I applied to 6sense through a referral for a Staff Engineer Backend position, bringing over 8.5 years of full-stack experience. The first round was a high-level design interview.

The core problem presented was designing a system for executing long-running workflows, similar to Apache Airflow, but with a critical modification: certain parts of these workflows should only execute once at a given time per day. For instance, if the execution graph is A -> B -> C, then jobs of type B must run only once within a specific daily window.

My High-Level Design involved using event buses/Kafka for communication, complemented by a dedicated scheduler for handling these time-constrained jobs. The architecture included a graph service for storing workflow definitions, an executor service responsible for running workflows and maintaining their states (backed by DynamoDB or a SQL database depending on scale), and scalable ephemeral workers (like Lambda or Go threads) to execute jobs ready for processing. Kafka was primarily envisioned as the event bus for triggering and coordinating tasks.

I designed the system with horizontal scalability in mind for every service, event bus, and database. Latencies would be controlled by scaling instances, and an API gateway layer would handle authorization and rate limiting. However, the interviewer provided crucial feedback on areas I missed or could have improved:

1. Input Validation: I did not discuss input validation, specifically detecting cyclic graphs, even when asked about sample JSON-based inputs. The interviewer implied this was expected, especially since they had solved it previously.
2. Kafka Specifics: While I proposed Kafka, I wasn't entirely clear on the nuances between topics and partitions, particularly regarding topic overheads. I acknowledged my primary experience was with RMQs.
3. Scheduler Optimization: The scheduler was considered an over-optimization for 'once a day' granularity, suggesting that simple database polling could suffice. I argued for its extensibility for future requirements like executing jobs at random times.

In hindsight, it felt like I was expected to align precisely with the interviewer's preconceived solution. Ultimately, I was ghosted after this round.