Changes in data delivery for UMGC users
Starting in October 2021, the UMGC will be delivering data for UMN users with MSI accounts to MSI's Shared User Resource Facility Storage (SURFS). Delivered data will be located in the "data_delivery" folder in your group's folder on MSI's primary filesystem (/home/GROUP/data_delivery/umgc). MSI does not charge for SURFS storage costs, but files expire and are removed one year after they've been delivered. Files should be copied to other MSI storage locations such as Tier2, Tier3, or your group's "shared" folder before they expire. Your group’s data_release folder will no longer be used for delivery of data, but existing files in data_release will remain and continue to follow the policy of expiring and being removed after five years. More details are available at z.umn.edu/surfs.
Data Release
There are four options for transferring data from the UMGC to clients: 1) delivery to the Minnesota Supercomputing Institute’s (MSI) high-performance file system, 2) download from a secure website, 3) download with Globus, or 4) shipment on an external hard drive. Please indicate your data delivery preference when placing an order for sequencing.
1. MSI storage
Internal clients have their data released to MSI's Shared User Resource Facility Storage (SURFS). Delivered data will be located in the "data_delivery" folder in your group's folder on MSI's primary filesystem (home/GROUP/data_delivery/umgc). MSI does not charge for SURFS storage costs, but files expire and are removed one year after they've been delivered. Files should be copied to other MSI storage locations such as Tier2, Tier3, or your group's "shared" folder before they expire.
2. Web download
Internal clients that opt-out of MSI storage and external clients can download their data from a secure website using either a web browser or a command-line download tool, complete instructions are provided in an email from the UMGC. The client’s data is available for download for 30 days, after which the data will be removed from the data download website and the client takes responsibility for storing the data.
3. Globus
Internal and External clients can use Globus to download their data. This is the recommended method for external clients to download large datasets.
4. Hard drive
External clients may have data shipped on a hard drive purchased by the UMGC and invoiced to the client at a cost of $250 per hard drive.
Data Recovery
The UMGC archives most customer data for a year and some datasets are retained for 5 years or more. If you need a dataset re-delivered email a request to next-gen@umn.edu to initiate data recovery. The UMGC does not provide any guarantee that data can be successfully recovered from the archive.