BICP, a San Diego-based consulting firm specializing in AI, Data & Analytics, is seeking an experienced Agentic AI Forward Deployed Engineer to join our team in an embedded capacity with one of our strategic clients in the apparel and retail industry. This is a fully onsite, forward-deployed role embedded with the client’s team in Carlsbad, California, five days per week.
This role is embedded directly with business, product, data science and technology stakeholders. You will translate high-value business problems into working agentic products, moving rapidly from discovery and prototype through production deployment and adoption.
You will partner closely with an Agentic AI Product Manager and Principal AI Platform Architect while helping experienced internal engineers and data scientists develop practical agentic AI capabilities. This is not a research-only, advisory or traditional machine-learning role. Success requires someone who can work through ambiguity, write production-quality code, integrate agents with enterprise systems and remain accountable for whether the resulting product delivers measurable business value.
Key Responsibilities
Build Agentic AI Products
- Design, develop and deploy production-grade AI agents that reason across multistep workflows, access enterprise data, invoke tools and APIs, and take actions within defined operating boundaries.
- Translate business requirements into working AI products through rapid discovery, prototyping, testing and iterative delivery.
- Build agentic workflows using the appropriate combination of deterministic logic, LLM reasoning, retrieval, tool calling and human approval.
- Develop single-agent and multi-agent solutions where warranted by the use case.
- Create reusable components, services and implementation patterns that accelerate subsequent AI products.
- Take direct ownership of implementation—not simply technical design or vendor management.
Connect AI to Enterprise Data and Systems
- Integrate agents with APIs, databases, SaaS platforms and internal business applications.
- Build secure retrieval and knowledge capabilities across structured and unstructured enterprise data.
- Develop and maintain RAG pipelines, embeddings, vector and hybrid search, metadata filters and grounding mechanisms.
- Integrate AI products with cloud data platforms such as Snowflake, Databricks and Azure-based services.
- Implement reliable tool interfaces with appropriate authentication, authorization, validation, retries and error handling.
- Work with Model Context Protocol and other emerging standards where they provide practical value.
Engineer for Production Reliability
- Build evaluation into the development lifecycle using representative test cases, golden datasets and automated regression testing.
- Measure task completion, answer quality, groundedness, tool-selection accuracy, latency, cost and escalation rates.
- Implement end-to-end logging, tracing, observability and production monitoring.
- Diagnose hallucinations, orchestration failures, retrieval issues and unexpected agent behavior.
- Design fallbacks, timeouts, retries, approval gates and graceful failure paths.
- Optimize model selection, prompts, context usage and architecture for quality, performance and cost.
Embed Security and Governance
- Apply least-privilege access controls to agents, tools and enterprise data.
- Implement human-in-the-loop approval for sensitive or consequential actions.
- Protect against prompt injection, unintended tool execution, sensitive-data exposure and data exfiltration.
- Maintain auditability across model interactions, tool calls, decisions and actions.
- Partner with architecture, security and governance teams to establish production-readiness standards.
- Ensure solutions meet enterprise requirements for privacy, compliance and responsible AI.
Work Directly With the Business
- Participate in discovery sessions with business stakeholders to understand workflows, pain points, decisions and desired outcomes.
- Demonstrate early working solutions and incorporate stakeholder feedback quickly.
- Explain technical tradeoffs and limitations clearly to nontechnical audiences.
- Challenge requests when agentic AI is not the appropriate solution.
- Help define measurable product outcomes rather than treating a functioning prototype as success.
- Operate effectively in a forward-deployed environment where requirements evolve through real-world use.
Mentor and Build Internal Capability
- Pair with internal engineers, data scientists and technical product team members who are developing agentic AI experience.
- Conduct architecture reviews, code reviews and technical working sessions.
- Teach practical patterns for agent design, retrieval, evaluation, security and production deployment.
- Document reusable components, engineering standards and lessons learned.
- Transfer knowledge and ownership so internal teams can operate and extend the products.
- Contribute to a collaborative engineering culture focused on delivery, experimentation and continuous learning.
Required Qualifications
- 7+ years of professional software, data, machine-learning or AI engineering experience.
- Significant hands-on experience building applications in Python and integrating RESTful APIs and enterprise systems.
- Demonstrated experience taking generative AI applications from prototype into production.
- Direct experience building at least one production agentic system involving multistep execution, tool use or autonomous workflow orchestration.
- Strong understanding of LLMs, prompting, structured outputs, tool calling, RAG, embeddings and vector search.
- Experience with agent orchestration, state and context management, memory, approval workflows and failure recovery.
- Experience establishing evaluation, observability and monitoring for nondeterministic AI systems.
- Strong knowledge of authentication, authorization, secrets management and secure enterprise integration.
- Experience deploying applications within modern cloud environments.
- Ability to make pragmatic build-versus-buy and deterministic-versus-agentic decisions.
- Strong communication skills and demonstrated success working directly with both technical and business stakeholders.
- Experience mentoring engineers or data scientists in an unfamiliar or rapidly evolving technology.
Preferred Qualifications
- Experience with Azure OpenAI, Azure AI services or comparable managed AI platforms.
- Experience integrating with Snowflake, Databricks or similar enterprise data environments.
- Familiarity with agent frameworks such as LangGraph, Semantic Kernel, OpenAI Agents SDK, AutoGen, CrewAI or comparable technologies.
- Experience with low-code or commercial agentic AI platforms and the ability to integrate them without creating unnecessary vendor dependence.
- Experience implementing Model Context Protocol servers or clients.
- Familiarity with model gateways, prompt and model versioning, LLMOps and AI governance.
- Experience deploying containerized services and event-driven or asynchronous workflows.
- Background in retail, apparel, consumer products, merchandising, planning, supply chain, marketing or customer experience.
- Experience operating in a consulting, customer-facing or forward-deployed engineering environment.
What Strong Performance Looks Like
- Within the first several months, the Forward-Deployed Agentic AI Engineer will:
- Develop a clear understanding of the company’s business priorities, data environment and technology ecosystem.
- Partner with product and architecture leadership to select an initial high-value, technically feasible use case.
- Move that use case from discovery through a working pilot and toward production.
- Establish repeatable engineering patterns for tool integration, evaluation, observability, security and human oversight.
- Help determine which capabilities should come from existing platforms and which should be built internally.
- Build trusted relationships with business stakeholders by connecting technical delivery to measurable outcomes.
- Increase the confidence and practical agentic AI capability of the internal engineering and data science teams.
Ideal Candidate
- The ideal candidate combines the instincts of a product-minded engineer, the technical depth of an applied AI practitioner and the adaptability of a consultant working inside the client environment.
- You are comfortable starting with an ambiguous business problem, sitting alongside stakeholders to understand how the work actually happens, and turning that understanding into a secure and reliable production product.
- You move quickly without confusing speed with shortcuts, and you understand that the goal is not to build the most sophisticated agent - it is to deliver the simplest dependable solution that creates meaningful business value.
About BICP
BICP is a San Diego-based AI, Data & Analytics consultancy that helps organizations close the gap between strategic priorities and the specialized capabilities required to execute them. Our Forward Deployed practitioners work alongside client teams, bringing deep expertise across AI/ML, Data, Analytics, Product, and Architecture to move initiatives from idea to implementation, production, and measurable business impact.