What is a batch job?
A batch job is a scheduled program that is assigned to run on a computer without further user interaction. Batch jobs are often queued up during working hours, then executed during the evening or weekend when the computer is idle.
Once the batch job is submitted, the job enters into a queue where it waits until the system is ready to process the job. If the job queue contains many jobs waiting to be processed, the system processes the jobs either in chronological order or by priority.
Batch jobs are frequently used to automate tasks that need to be performed on a regular basis, like payroll, but don’t necessarily need to occur during the day or have an employee interacted with the system. Jobs that happen on a regular basis are incorporated into batch schedules.
A job runs one or more instances of your executable code. Unlike applications, which handle HTTP requests, jobs are designed to run one time and exit. When you create a job, you can specify workload configuration information that is used each time that the job is run.
The most valuable benefits of batch jobs include:
- Large programs can utilize more dedicated servers when the work processes are in night mode.
- With fewer users or employees online during off hours, the performance will be faster and more efficient at night. During the day it could be restricted to fewer servers and numbers needed.
- Employees are freed up to focus on less repeatable, more creative tasks.
The following guide will walk you through creating a Code Engine project and application using sample code, and then you will create batch jobs and run them. The following steps can also be followed to deploy an application to Code Engine from source code after following the setup instructions in Next Steps.
Happy Hacking!
Let's Build Something
Steps
- Sign Up for a Free IBM Cloud Account
- Install the IBM Cloud Developer Tools CLI (command line interface, the commands you type into your Terminal)
- Install the IBM Cloud Code Engine Plugin
- Login to Your IBM Cloud Account via the CLI
- Create a Code Engine Application
- Create a Job from a Public Repo
- Run a Batch Job
- NEXT STEPS! Build a Container Image from Source Code to Deploy to Code Engine and Run Batch Jobs
Setup & Installation
2. Install the IBM Cloud Developer Tools CLI
- For Mac and Linux, run the following command in Terminal:
curl -sL https://raw.githubusercontent.com/IBM-Cloud/ibm-cloud-developer-tools/master/linux-installer/idt-installer | bash
- Verify the IBM Cloud Developer Tools CLI is installed
ibmcloud dev help
- For Windows 10 Pro, run the following command as an administrator in Powershell:
[Net.ServicePointManager]::SecurityProtocol = "Tls12, Tls11, Tls, Ssl3"; iex(New-Object Net.WebClient).DownloadString('https://raw.githubusercontent.com/IBM-Cloud/ibm-cloud-developer-tools/master/windows-installer/idt-win-installer.ps1')
Note For Windows Users: If you encounter a Git error similar to the one below, you will need to install Git in the correct path.
bash: git: command not found
Follow the Windows Guide HERE
3. Install the IBM Cloud Code Engine Plugin
- For Mac, Linux, and Windows 10 Pro, run the following command:
ibmcloud plugin install code-engine
- Verify the IBM Cloud Code Engine Plugin is installed
ibmcloud ce help
4. Login to Your IBM Cloud Account via the CLI
- For Mac, Linux, and Windows 10 Pro, run the following command:
ibmcloud login
Enter email and password
View available resource groups
ibmcloud resource groups
- Assign a target resource group (default to your "Default")
ibmcloud target -g Default
Deploy to Code Engine & Run Batch Jobs
5. Create a Code Engine Application
- Create a new Code Engine project and give it a name
ibmcloud ce project create --name PROJECT_NAME
ibmcloud ce project create --name sandbox
- Create a new app from a sample Container Image
ibmcloud ce application create --name APP_NAME --image IMAGE
ibmcloud ce application create --name myapp --image docker.io/ibmcom/hello
- Check the application status
ibmcloud ce application get -n APP_NAME
ibmcloud ce application get -n myapp
- Get the live application URL
ibmcloud ce application get -n APP_NAME -output url
ibmcloud ce application get -n myapp -output url
- View the live application at the URL in your browser
6. Create a Job from a Public Repo
- Create a job configuration that is named myjob and uses the container image docker.io/ibmcom/firstjob
ibmcloud ce job create --name JOB_NAME --image IMAGE
ibmcloud ce job create --name myjob --image ibmcom/firstjob
Note: the format for creating a new job from an image is ibmcom/firstjob from the image docker.io/ibmcom/firstjob.
7. Run a Batch Job
- Run a job
ibmcloud ce jobrun submit --name testjobrun --job myjob --array-indices "1 - 5"
Note: the following jobrun submit command creates five new instances to run the container image that is specified in the myjob job. the resource limits and requests are applied per instance, so each instance gets 4 G memory and 1 vCPU. This job allocates 5 * 4 G = 20 G memory and 5 * 1 vCPU = 5 vCPUs.
- Resubmit a job run based the configuration of a previous job run, use the jobrun resubmit command
ibmcloud ce jobrun resubmit --jobrun testjobrun
- Access the Job Details, including the status of your instances, configuration details, and environment variables of your job
ibmcloud ce job get --name myjob
Options for Creating and Running Jobs
CONGRATULATIONS!
If you've made it this far, then Kudos to you! The example commands given can be followed to create an application from a container image and deploy it to Code Engine with your own images. You can also follow the example commands given to create and run batch jobs with your own public images and repositories. For more information to get started with your own container images and/or source code, check out the Tutorial Tuesday Code Engine Guide along with the Code Engine Documentation.
Top comments (0)