About the role

Role: AI Engineer

Crogl is building a fully autonomous AI team member for security operations (SOC). You’ll design, evaluate, and improve agentic systems that run in production and help security teams investigate and respond to threats more effectively.

Responsibilities

Build LLM-powered features, workflows, and agentic systems for real security use cases.
Design and implement evaluation frameworks to measure:
- agent quality and accuracy
- reliability and trustworthiness
- business impact
Create automated benchmarks, regression tests, and evaluation datasets.
Investigate agent failures and develop systematic improvements.
Experiment with prompting, tool use, retrieval, memory, planning, and reasoning strategies.
Build infrastructure for rapid experimentation, evaluation, deployment, and monitoring.
Collaborate with customers and internal teams to understand workflows and identify automation opportunities.
Contribute to best practices for testing, observability, and production reliability.
Stay current with advances in LLMs, agents, and AI evaluation.

Core Focus: Evaluating AI Systems

Build and maintain evaluation systems that answer questions such as:
- Are agents producing accurate investigations?
- Do prompt/tool/model changes improve outcomes?
- How are regressions detected before customers notice?
- Which failure modes matter most?
Develop benchmarks, datasets, automated evaluations, regression pipelines, and observability systems.

Requirements

Strong programming skills, preferably Python.
Solid software engineering fundamentals: testing, debugging, and system design.
Experience building LLM-based applications/projects/products using modern AI tools.
Ability to design experiments, interpret results, and make data-driven decisions.
Strong communication skills and cross-disciplinary collaboration.
Curiosity, ownership, and fast learning.

Nice to Have / Stand Out

Experience building AI agents, copilots, or workflow automation.
Experience designing evaluation/benchmark/testing frameworks for AI systems.
Familiarity with LLM ecosystems such as OpenAI, Anthropic, Gemini, or open-source.
Experience with retrieval systems, vector databases, and RAG.
Familiarity with LangGraph, OpenAI Agents SDK, MCP, or similar agent frameworks.
Experience with observability, tracing, and production monitoring for AI.
Exposure to cybersecurity, security operations, or developer tooling.
Open-source contributions, research projects, or personal AI products.

About Crogl, Inc.

Crogl, Inc. builds AI-powered systems for security teams. The company focuses on practical applications of large language models, agent architectures, and security expertise to help practitioners investigate, understand, and respond to threats faster.

Tags