Need for a product to handle Past and Future Batch workload types

As far as i am concerned Batch processing is the best example for need of elasticity of infrastructure. Whether it is ETL, File processing, Event based, Time based or Ingress request based. Lot of Organizations have full set of VM's hogging on to the computes to run a job once in a day, week or month. Lot of companies have mainframe batch jobs, windows task schedulers, linux Cron jobs, in house job schedulers etc.,
If you are looking to future, K8's Cron job, Volcano or Kube Batch can offer nice features to maintain batch job workloads in Kubernetes world. Serverless can be an alternative too. But the question is what is the difference between 0 VM's vs 1 worker VM. Because you are NOT going to save a lot by going to Zero. And if you are using something like Azure Functions as your Serverless strategy you need to understand, if you are really leveraging consumption model or NOT. Don't come to the assumption if you are using Azure Functions you are serverless.
The questions is how to transform thousands of batch jobs you have on VM's currently into a K8's or serverless without doing it from the scratch. I feel there is a niche here. If we have a product / Offering that can seamlessly migrate from VM based batch workloads into K8's / Serverless that will be great saving and step in the right direction for lot of companies in Erasing tech debt.
The product should give an option to control all the batch work loads in the org to be controlled through Same portal with an abstraction layer. That way you are still using the same portal if you are running a mainframe job or Windows task scheduler. Then the trick is to extend that portal functionality to K8's jobs as well. Underlying execution plane could be whatever you chose (K8 cron jobs or Kube batch).
Now users can run their jobs through the portal to their normal execution engine and beta test on K8's execution engine , get comfortable with it. Later they can shutdown their jobs in Linux VM and start running them in K8's. This way there will be seamless adoption and transition. This will help organization to entice more developers into K8's world.

Comments

Post a Comment

Popular posts from this blog

Dhirubhai Notorious or Famous???????

This article is about a legend called Dhirubhai. Ambani The collected information about Reliance founder Dhirubhai from book Polyster Prince (which is banned in INDIA) and from internet. These facts are eye openers for rotten politics or eye brow raisers for public or shot in the arm for Indian system or brain teasers for Tax / Finance authorities....... Whatever it is, a man dreamed for something achieved it, the path he chosen might be controversial but not the hard work he put for it, neither his capabilities nor his thoughts..... One thing for sure 2 generations before me they dont know about this guy , might be no body discusses about him after 2 generations, i am proud to be from the same historic period as he is..... His famous quotes: Think big, think fast, think ahead. Ideas are no one's monopoly "We cannot change our Rulers, but we can change the way they Rule Us. "Changing orbits is the key to our progress" Some Unofficial statements made by him : I dont
Read more

Sri Sri kavitha

Hi All, (From Sri Sri Mahaprashtanam.......) Nenokkadini Ontarini na gadilopala chikatilo , chikatilo na gadilo, na gadilo chikatilo... chikatilopala na gadi lo..... Nenokkadini nidraku veliyai kannulu nindina kaviri to , kaviri nindina kannulatho, kannulalo kaviri to..... Nenokkadini nidraku veliyai gundalu kappina kumpatitho, kumpati kappina gundelatho , gundalalo kumpati tho, kumpatilo gundelatho.... naa kanu godal, na kanu gonal baakulatho , baakulu kappina baatalatho, baatalalo baakulatho......
Read more

Excerpt from the book "Made to Stick"

Image
This is one of my favorite books this year. The authors in this book talk about 6 principles to make messaging stick.  S:  Simple: Find the core and share the core U: Unexpected: Get attention(Surprise), Hold Attention (Interest). C:  Concrete: Help people understand, remember and coordinate. C:  Credible: Help people Believe. External and internal credibility. E:  Emotional: Make people care through the power of association, self-interest, and appeal to Identity. S:  Stories: Get people to Act through simulation and inspiration. Golden Rule of Simplicity: A one-sentence statement so profound that an individual could spend a lifetime learning to follow it. People can be driven to irrational decisions by too much complexity and uncertainty. Simple messages are core and compact. We need to shift our thinking from "What information do I need to convey? to "What questions do I want my audience to ask ?" Curiosity comes from gaps in our knowledge. Statistics will, and should,
Read more