Minimizing Service Costs for Batch Processing Jobs

C.

The best option for minimizing service costs while running nightly batch processing jobs that take around 2 hours to complete is to use Compute Engine with preemptible VM instances of the appropriate standard machine type.

Preemptible VM instances are instances that can be used at a much lower cost than regular instances but can be shut down at any time by Google with just a 30-second notice. Since batch processing jobs are non-interactive and can tolerate interruptions, preemptible instances can be a cost-effective option. These instances are suitable for short-lived, fault-tolerant, and batch jobs that can withstand instance preemption.

Selecting a VM instance type that supports micro bursting can also be useful in case there is a temporary surge in processing requirements. This can help ensure that the processing jobs are completed within the desired time frame while minimizing costs.

On the other hand, selecting Google Kubernetes Engine (GKE) for running batch processing jobs may not be the most cost-effective solution. While GKE provides several benefits such as automatic scaling, load balancing, and container orchestration, it may not be necessary for a simple batch processing job that runs nightly. Additionally, using a single-node cluster with a small instance type may not provide the required compute resources for running the batch jobs, while using a three-node cluster with micro instance types may be an overkill and unnecessarily increase costs.

In summary, the best option for minimizing service costs while running nightly batch processing jobs that take around 2 hours to complete is to use Compute Engine with preemptible VM instances of the appropriate standard machine type.