We are looking for a Senior Software Engineer – Lakehouse & Applied AI with strong hands-on experience in Databricks, PySpark, Python, cloud infrastructure, and GenAI/LLM technologies .
The ideal candidate will combine data engineering and software engineering expertise with experience building production-grade AI/LLM solutions, cloud platforms, infrastructure-as-code, and containerized workloads .
This role involves owning solutions end-to-end, from discovery and rapid prototyping through production deployment , while working directly with stakeholders in a forward-deployed environment.
Key Responsibilities
- Design and deliver end-to-end lakehouse and applied-AI solutions from prototype through production.
- Build scalable data platforms using Databricks, PySpark, Delta Lake, and Azure .
- Develop agentic AI workflows and tool-using AI agents for software and infrastructure automation.
- Build AI solutions using LLMs, embeddings, RAG, and MCP or similar tool-serving frameworks .
- Develop reusable multi-tenant CI/CD and deployment frameworks .
- Build and deploy local/edge LLM and embedding-serving environments .
- Develop Python-based REST APIs and microservices using FastAPI, Flask, or similar frameworks.
- Deploy and orchestrate production workloads using Docker, Kubernetes, ECS/Fargate, and Airflow .
- Manage cloud infrastructure using Terraform and Terragrunt across Azure and AWS.
- Work with services including Azure Databricks, ADLS Gen2, Key Vault, AWS ECS, S3, RDS, SQS, and SSM .
- Develop AI evaluation and testing frameworks covering regression detection, retrieval quality, task completion, and safe rollback .
- Implement monitoring and observability using tools such as Prometheus and Grafana .
- Work directly with customers and stakeholders to translate ambiguous requirements into production-ready technical solutions .
Required Qualifications
- 6+ years of experience in software/data engineering.
- Strong hands-on experience with Databricks, PySpark, Delta Lake, and Python .
- Experience with Medallion Architecture, Auto Loader, Unity Catalog, and Databricks Asset Bundles .
- Hands-on experience with GenAI, LLMs, AI agents, RAG, embeddings, or agentic workflows .
- Working knowledge of MCP or similar LLM tool-serving frameworks .
- Strong experience with Terraform and Terragrunt .
- Experience across Azure and/or AWS cloud environments .
- Hands-on experience with Docker, Kubernetes, ECS/Fargate, and CI/CD .
- Strong Python development skills with FastAPI, Flask, REST APIs, and microservices .
- Experience with Airflow or similar workflow orchestration tools .
- Knowledge of Prometheus, Grafana, or similar observability platforms .
- Strong problem-solving skills with the ability to own solutions from prototype to production .
- Comfortable working in a customer-facing, forward-deployed engineering environment .
Preferred Qualifications
- Experience with local/on-premises LLM deployment using Ollama, Llama, Hugging Face, or similar technologies.
- Experience working with Snowflake alongside Databricks .
- Experience building multi-tenant data or AI platforms .
- Telecommunications experience involving 3G/4G-LTE OSS/BSS, network management, backhaul, or 5G .
- Experience with AI-assisted software engineering and autonomous/agentic development workflows .
- Experience optimizing production systems for latency, reliability, scalability, and cost .
ABOUT BRICKRED SYSTEMS
BrickRed Systems is a global leader in next-generation technology consulting and workforce solutions, specializing in delivering high-quality talent across digital, engineering, marketing, analytics, finance, operations, and business transformation domains. With a strong emphasis on innovation, scalability, and client success, BrickRed Systems helps organizations solve complex business challenges by providing skilled professionals across strategy, technology, creative, and operational functions. BrickRed fosters a culture of continuous learning, collaboration, and excellence, enabling professionals to contribute to high-impact global initiatives while advancing their careers.