- {
- “cells”: [
- {
“cell_type”: “code”, “execution_count”: 1, “metadata”: {}, “outputs”: [], “source”: [
“import mdf_toolbox”]
}, {
“cell_type”: “markdown”, “metadata”: {}, “source”: [
“# Globus Transfer Utilitiesn”, “While using the Globus Web App to perform data transfers is easy, programmatically doing the same has some overhead, which these utilities try to reduce.”]
}, {
“cell_type”: “markdown”, “metadata”: {}, “source”: [
“## quick_transfern”, “quick_transfer() starts a transfer and monitors its progress. You must have a TransferClient already created and authenticated (see the Authentication Utilities tutorial).n”, “n”, “You must provide:n”, “n”, “transfer_client: An authenticated Transfer clientn”, “n”, “source_ep: The source endpoint IDn”, “n”, “dest_ep: The destination endpoint IDn”, “n”, “path_list: A list of tuples, where the first element is a source path,n”, ” and the second element is the destination path.n”, ” Directory paths must end in a slash, and file paths must not.”]
}, {
“cell_type”: “code”, “execution_count”: 2, “metadata”: {}, “outputs”: [], “source”: [
“transfer_client = mdf_toolbox.login(services=[“transfer”])[“transfer”]n”, “source_ep = “”n”, “dest_ep = “”n”, “path_list = [n”, ” (“/home/source/files/file.dat”, “/home/destination/doc.dat”),n”, ” (“/home/source/all_reports/”, “/home/destination/docs/reports/”)n”, “]”]
}, {
“cell_type”: “code”, “execution_count”: null, “metadata”: {
“scrolled”: true}, “outputs”: [], “source”: [
“mdf_toolbox.quick_transfer(transfer_client, source_ep, dest_ep, path_list)”]
}, {
“cell_type”: “markdown”, “metadata”: {}, “source”: [
“It is normal for Globus Transfer to encounter a few transient errors when transferring data. The transfer is very robust and can usually recover from most issues. By default, quick_transfer() will tolerate 10 error events before cancelling the transfer. You can customize this with retries, with -1 indicating infinite retries (until the transfer times out naturally).n”, “n”, “Additionally, you can customize the poll interval (in seconds) with interval.”]
}, {
“cell_type”: “code”, “execution_count”: null, “metadata”: {}, “outputs”: [], “source”: [
“mdf_toolbox.quick_transfer(transfer_client, source_ep, dest_ep, path_list,n”, ” interval=7, retries=-1)”]
}, {
“cell_type”: “markdown”, “metadata”: {}, “source”: [
“## custom_transfern”, “custom_transfer() is similar to quick_transfer(), but allows you greater control over the finer details. It is also more complex.n”, “n”, “All of the required parameters from quick_transfer() are required for custom_transfer(). interval is also available. However, you can additionally set the natural timeout due to inactivity (in seconds) with inactivity_time.n”, “n”, “The major difference between the two transfer utilities is that custom_transfer() will yield any error events and allow you to .send(False) to cancel the transfer.”]
}, {
“cell_type”: “code”, “execution_count”: null, “metadata”: {}, “outputs”: [], “source”: [
“transfer_generator = mdf_toolbox.quick_transfer(n”, ” transfer_client, source_ep, dest_ep,n”, ” path_list, inactivity_time=600)n”, “res = next(transfer)n”, “print(res)n”, “try:n”, ” while True:n”, ” res = transfer.send(True)n”, ” print(res)n”, “except StopIteration:n”, ” pass”]
}, {
“cell_type”: “markdown”, “metadata”: {}, “source”: [
“## get_local_epn”, “get_local_ep() attempts to discover the local machine’s Globus Connect Personal endpoint ID. It does not properly detect Globus Connect Server endpoints, and can sometimes return inaccurate results if the local machine is not running GCP. If there are multiple possible endpoints, the user is prompted to choose the correct one.n”, “n”, “get_local_ep() requires a TransferClient.n”, “n”, “If possible, it is recommended to get the local EP ID some other way, such as through the Globus Web App or by asking the endpoint’s owner.”]
}, {
“cell_type”: “code”, “execution_count”: null, “metadata”: {
“scrolled”: true}, “outputs”: [], “source”: [
“mdf_toolbox.get_local_ep(transfer_client)”]
}, {
“cell_type”: “code”, “execution_count”: null, “metadata”: {}, “outputs”: [], “source”: []}
], “metadata”: {
- “kernelspec”: {
- “display_name”: “Python 3”, “language”: “python”, “name”: “python3”
}, “language_info”: {
- “codemirror_mode”: {
- “name”: “ipython”, “version”: 3
}, “file_extension”: “.py”, “mimetype”: “text/x-python”, “name”: “python”, “nbconvert_exporter”: “python”, “pygments_lexer”: “ipython3”, “version”: “3.6.4”
}
}, “nbformat”: 4, “nbformat_minor”: 2
}