> For the complete documentation index, see [llms.txt](https://cgps.gitbook.io/pathogenwatch/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://cgps.gitbook.io/pathogenwatch/how-to-use-pathogenwatch/uploads-and-folders/genome-uploads-folders.md).

# Genome Uploads & Projects

## Introduction

Free of charge, we provide (to signed-in users) the ability to upload and analyse large numbers of microbial genomes in Pathogenwatch. 

Genomic data & metadata can be uploaded on the [Upload Page](#upload-page), where they get uploaded into "projects". Data can be manually organised on the [Projects Page](#folders-page).

## Upload Page

To upload your own microbial pathogen genome data, open the Upload Page and follow the onscreen instructions. Files can be either dragged and dropped onto this page, or added by clicking on the <img src="/files/YbZYNg548PCK2U7xsvkn" alt="" data-size="line"> plus button (bottom right) and manually selecting files. There are several [File Formats](#file-formats) available to you.

Each upload event will automatically create a new project, containing the uploaded genomes, which can later be accessed & reorganised on the Projects Page.

The Upload Page is accessible from both the banner menu and side (hamburger) menu.&#x20;

<div><figure><img src="/files/SNDUiVlWg7cODlryP3Md" alt="" width="318"><figcaption></figcaption></figure> <figure><img src="/files/jPn3NuN9d3HdsWPhhRPq" alt=""><figcaption></figcaption></figure></div>

## File Formats

You will have the choice to upload three types of  genome sequence file:

1. [Single Genome FASTA](#single-genome-fastas) (.fasta) files: Each file contains a single genome (i.e. bacterial genomes)
2. [Multi-Genome FASTA](#multi-genome-fastas) (.mfa) files: Each file contains multiple genomes, with one genome per record/contif (e.g. viral genomes)
3. [Paired read FASTQ](#paired-read-sequence-fastqs) (.fastq.gz) files: Pairs of read files in FASTQ format (compressed)

You can also upload an accompanying [CSV](https://en.wikipedia.org/wiki/Comma-separated_values) format file containing metadata about the genomes.

1. [Metadata](#metadata) (.csv) files: recommended (but optional) metadata records for each genome in CSV format.

### Single genome FASTAs

Sequences must be represented in standard IUPAC code (i.e. `ATCGATCGNA`). Each record represents a single contig in the assembly. The file name is used to name the genome by default and to link to a record in an accompanying metadata CSV. More than one can be uploaded at a time, though we recommend small batches on slow or unstable internet connections.

### Multi-genome FASTAs

Sequences must be represented in standard IUPAC code (i.e. `ATCGATCGNA`). Each record represents an assembled or complete viral genome. The record header will be used to name each genome by default and to link to records in an accompanying metadata CSV. More than one can be uploaded at a time, though we request users to be mindful not to submit thousands of genomes at once as it will impact other users.

### Paired Read Sequence FASTQs

You can also upload a limited number of pairs of FASTQ files for assembly using our in-house assembly pipeline. The default genome name is taken from the shared part of the filename of the FASTQs. For more details about this pipeline, please see the [technical documentation](/pathogenwatch/technical-descriptions-of-analysis-tools/genome-assembly/short-read-assembly.md).

## Metadata

Optional metadata files are accepted in CSV format, with a `.csv` file ending. Including as much information as possible will enhance the investigations possible using the [Interactive Collection View](/pathogenwatch/how-to-use-pathogenwatch/collections/using-the-icv.md).&#x20;

{% hint style="success" %}
We strongly recommend including when and where the sample was taken in order to get the maximum use from Pathogenwatch.
{% endhint %}

You can download templates and examples below. Only the filename is a required field as it is used to link the record to the FASTA/FASTQ files.

{% file src="/files/Pj6kyUIlm0U2wH2vtxF1" %}

{% file src="/files/eIJ5eWwV11HCO9XCM0kH" %}

{% file src="/files/3GFgEvAwb6eLY8AtKRYN" %}

{% file src="/files/GhmRKsOPp5K7Xv0Quk1D" %}

<table><thead><tr><th width="127.6875">Field(s)</th><th width="460.22265625">Purpose</th><th></th></tr></thead><tbody><tr><td><em>filename</em></td><td>The uploaded FASTA file for this record.</td><td>bswp.fasta</td></tr><tr><td><em>displayname</em></td><td>The sample name for display in Pathogenwatch.</td><td>Water_Pump_1</td></tr><tr><td><em>latitude</em><br><em>longitude</em></td><td>Geographical location - accepts ISO 6709 decimal notation, and is used to infer the country location of the sample.</td><td>51.5133228<br>-0.1392891</td></tr><tr><td><em>country</em><br><em>county</em></td><td>As an alternative to providing latitude or longitude, country and country subdivision <a href="https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes">ISO 3166</a> codes can be provided instead, and a standard latitude/longitude will be inferred. The county field will be checked first, and then the country field. Two or three letter ISO 3166-1 country codes can be used.</td><td>GB<br>GB-WSM</td></tr><tr><td><em>year</em><br><em>month</em><br><em>day</em></td><td>Sample timestamps are recorded as three separate columns: <strong><code>year</code></strong>, <strong><code>month</code></strong>, <strong><code>day</code></strong>. You can include year, year-month, or year-month-day.  Year-month-day are the dates used as the "Sample date" in the Genome Browser <a href="/pages/P8b4ztIUw72rv41RXihc#using-filters">filters</a>. Other date columns are acceptable as metadata, but only year, month, day will be used in the <a href="/pages/o5jy2ZAcx6jHsP5stiE3">Timeline</a>.</td><td>1854<br>9<br>3</td></tr><tr><td><em>literatureLink</em><br><em>pubmed</em><br><em>doi</em></td><td>Literature references can be provided as <a href="https://www.doi.org/">DOI</a> system identifiers or <a href="https://pubmed.ncbi.nlm.nih.gov/">Pubmed</a> identifiers. Only one citation can be used per sample, with the <code>literatureLink</code> column checked first, followed by <code>pubmed</code> and then <code>doi</code>. The identifiers can be optionally prepended with <code>pmid</code> and <code>doi</code>.</td><td>29647347<br>pmid:29647347<br>10.1093/ije/dyt193</td></tr></tbody></table>

{% hint style="danger" %}
You are responsible for ensuring that personally identifiable information (e.g. email addresses), or other restricted data (e.g. patient identifiers), is not stored on our servers, or is in accordance with the relevant laws and regulations.

If you wish to view private data within the Pathogenwatch collection viewer, we provide an offline viewing service that keeps your metadata on your local computer - see [Using the Interactive Collection View](/pathogenwatch/how-to-use-pathogenwatch/collections/using-the-icv.md).
{% endhint %}

## Uploading Tips

#### Upload Files Individually

If your connection regularly disconnects, then uploading files individually will increase the chance that each file will be uploaded successfully. Compressing your files will also help.

#### Monitoring Upload Progress

When you upload files, you are automatically brought to that upload's Project Viewer page to monitor the upload process. The tasks being carried out, and their individual progress are tracked on the screen, both in the animated circle and in the upload dialogue box.

As results arrive from [Speciator](/pathogenwatch/technical-descriptions-of-analysis-tools/speciator.md), and then [MLST](/pathogenwatch/technical-descriptions-of-analysis-tools/lineage-and-genotyping/mlst.md), the species and type are displayed for each submitted genome in the animated circle, along with the status of the upload.&#x20;

Once all tasks are complete, you can click the "View Genomes" button in the centre of the circle (or click the "Summary" button) to view the uploaded genomes in a tabular format, similar to the display on the "Genomes" page.

<figure><img src="/files/jc1Kke8U4UZjWDvdoLDT" alt=""><figcaption><p>in-progress upload screen</p></figcaption></figure>

{% hint style="info" %}
Uploaded genomes will be placed in a newly created [project](/pathogenwatch/how-to-use-pathogenwatch/uploads-and-folders/folders.md).
{% endhint %}