4-8 years production AI/ML experiencePythonPyTorch / TensorFlowLLM orchestration frameworksLLM evaluation & experimentationProduction AI shippingScalable backend / APIsLLM fine-tuningAutomated eval & benchmarking pipelinesRLHF / human-feedback optimizationRegulated industry experienceEarly-stage startup experience
Job Description
An early-stage AI startup is seeking an experienced Applied AI Engineer to build and deploy production AI applications powered by large language models. The role involves designing evaluation frameworks, optimizing LLM performance, developing scalable backend services, and collaborating with engineering and product teams. This position offers significant ownership over the company's AI platform with potential to evolve into a leadership role as the company grows.
Overview
An early-stage AI company building intelligent automation products for a highly regulated industry is seeking an Applied AI Engineer. This is a foundational role with significant ownership over the direction of the company's AI platform. As the company grows, this position is expected to evolve into a leadership role overseeing the organization's AI strategy.
Location
San Francisco Bay Area (Onsite, Full-Time)
What You'll Do
•Build, deploy, and optimize production AI applications powered by large language models
•Evaluate foundation models and determine the best solution for specific product use cases
•Design prompt strategies and iterate on them using structured testing and performance metrics
•Develop evaluation frameworks that measure quality, accuracy, reliability, and business outcomes
•Fine-tune language models using proprietary datasets when appropriate
•Design AI workflows that continuously improve through production data and human feedback
•Build scalable backend services and APIs that integrate AI capabilities into customer-facing products
•Collaborate with engineering and product teams to translate business problems into practical AI solutions
•Monitor production performance and continuously improve latency, reliability, and response quality
•Ensure AI systems meet security, privacy, compliance, and scalability requirements
•Stay current with emerging AI techniques, models, and tooling, recommending new technologies when they provide meaningful value
Required Qualifications
•4-8 years of professional experience building production AI or machine learning applications
•Demonstrated success shipping AI products used by real customers
•Strong Python development experience
•Experience with modern deep learning frameworks such as PyTorch or TensorFlow
•Experience building applications with LLM orchestration frameworks such as LangChain, LlamaIndex, DSPy, Instructor, or similar tools
•Experience evaluating and improving LLM performance through experimentation and testing
•Strong software engineering fundamentals, including systems design and scalable application development
•Ability to translate business objectives into practical AI solutions
•Excellent communication and collaboration skills
•Ability to work onsite in the San Francisco Bay Area
Preferred Qualifications
•Experience at an early-stage startup or high-growth technology company
•Experience fine-tuning language models using proprietary datasets
•Experience designing automated evaluation frameworks and benchmarking pipelines
•Experience deploying reinforcement learning or human-feedback optimization workflows
•Experience with conversational AI, voice technologies, or real-time AI applications
•Experience building AI systems within regulated industries
•Contributions to open-source AI projects, technical writing, conference speaking, or the broader AI community
Why Join
•Build AI products that solve meaningful, real-world problems
•Work directly with company leadership on technical strategy
•Own major architectural decisions across the AI platform
•Influence the future direction of AI within a rapidly growing company
•Opportunity to grow into a senior technical leadership role as the organization scales
•Competitive salary, equity, and comprehensive benefits
Resume Intelligence
Assess your experience against this job and tailor your bullet points instantly.
Sign in required
Please sign in to assess your resume alignment and generate custom copyable work bullets.