Scale AI System Design
Summary
I was presented with a challenging system design problem during my interview at Scale AI, which involved designing a scalable task processing system with integration to an external LLM service.
Full Experience
During my interview at Scale AI, I encountered a significant system design challenge. The problem focused on creating a robust and efficient system capable of managing tasks and jobs, and then processing these tasks through a third-party LLM service. This required careful consideration of data fetching from MongoDB, job creation for up to 5000 tasks, and synchronous interaction with the LLM service, handling batching of 10 tasks per API request.
Interview Questions (1)
Design a system that fetches tasks (represented as JSON blobs) from MongoDB, and lets operators create jobs. A job can be comprised of up to 5000 tasks. A task is completed by sending it to a 3rd party LLM service and waiting for a response (synchronously). Each API request to the 3rd party LLM service can contain 10 tasks.