services	platforms	author
batch	python	msonecode

Create Azure Batch PaaS Cloud Service working with Azure Data Lake Store by Python

Introduction

A get-started Azure Batch sample written in Python, using Cloud Service Configuration for computing nodes in Batch pool, using Data Lake Service.

Prerequisites

Python Tools for Visual Studio

Install Python Tools for Visual Studio.

http://aka.ms/ptvs

Python SDK

Python Recommend the latest 3.5.2 version

https://www.python.org/downloads/

Azure Storage and Azure Batch Python packages

Install packages with the below commands in cmd.

cd /d <The sample directory containing vs solution>
python -m pip install -r requirements.txt

Azure Batch account

Once you have an Azure subscription, create an Azure Batch account.

https://docs.microsoft.com/en-us/azure/batch/batch-account-create-portal

Azure Storage account

Create an Azure Storage account.

https://docs.microsoft.com/en-us/azure/storage/storage-create-storage-account#create-a-storage-account

Azure Data Lake Store

Create an Azure Azure Data Lake Store account.

https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-get-started-portal#create-an-azure-data-lake-store-account

Azure AD Application

Create an Azure AD Application and grant privilege to Data Lake Store.

https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-authenticate-using-active-directory

Azure Tenant Id

Get Azure Tenant Id.

https://blogs.technet.microsoft.com/heyscriptingguy/2013/12/31/get-windows-azure-active-directory-tenant-id-in-windows-powershell/

Build the Sample

Prepare the information at below from Azure portal.

A batch account

• The batch account name
• The batch account key
• The batch account URL

A storage account

• The storage account name
• storage account key

Azure Tenant Id

• The Azure Tenant Id

A Azure Data Lake account

• The Azure Data Lake name

A Azure AD Application

• The Azure AD Application Id
• The Azure AD Application Key

Replace the following properties in file python_tutorial_client.py with your actual values:

_BATCH_ACCOUNT_NAME = ''

_BATCH_ACCOUNT_KEY = ''

_BATCH_ACCOUNT_URL = ''

_STORAGE_ACCOUNT_NAME = ''

_STORAGE_ACCOUNT_KEY = ''

Replace the following properties in both python_tutorial_client.py and python_tutorial_task.py with your actual values:

_DATALAKE_NAME = ''

_TENANT_ID = ''

_AAD_APPLICATION_ID = ''

_AAD_APPLICATION_KEY = ''

Running the Sample

Execute python_tutorial_client.py and see the output displayed as below.