Senior Research Engineer Agentic Behavior

Senior Research Engineer Agentic Behavior Overview

JetBrains is looking for a Senior Research Engineer focused on agentic behavior to help improve how AI coding agents work with Kotlin across the company’s ecosystem. The team behind this role is responsible for understanding how AI agents read, generate, and refine Kotlin code across Android, Kotlin Multiplatform, server-side applications, web, desktop, and other environments. The work centers on building the infrastructure and research loops that measure agent performance on real Kotlin development tasks and then use those measurements to improve the behavior of the models and tools involved.

In this position, you would own the full improvement cycle: study where agents fail on Kotlin tasks, create evaluations that capture those failures, research and implement ways to address them, and then measure whether the changes actually help. The output of this work is intended to influence how developers experience Kotlin through AI coding assistants at scale.

What you would do

• Build systems for analyzing agent errors in a structured way, including tools that can capture, sort, and study the mistakes AI coding agents make when producing Kotlin code.
• Create observability pipelines that work over agent traces and help reveal patterns in how agents behave, using data from JetBrains IDEs, Junie, Claude Code, Cursor, and other coding agents.
• Design and maintain evaluation pipelines that measure Kotlin code generation quality across multiple dimensions, such as whether the code is correct, idiomatic, builds successfully, uses frameworks appropriately, and includes adequate test coverage.
• Develop simulation environments that let the team evaluate coding agents on realistic Kotlin development tasks, including starting new KMP projects, handling Gradle dependencies, and moving Spring applications from Java to Kotlin.
• Own the evaluation infrastructure end to end, including metrics, experiment tracking, automated regression testing, and reproducible benchmarking.
• Investigate and test post-training approaches such as supervised fine-tuning, DPO, and GRPO to improve model behavior on Kotlin-specific patterns, idioms, and frameworks.
• Explore context-engineering methods such as CLAUDE.md and AGENTS.md files, compiler-as-verifier feedback loops, Kotlin LSP integration, and MCP-based tooling.
• Design and run experiments to understand the effect of changes, using A/B comparisons, benchmark suites, and before-and-after analysis on real codebases.
• Collaborate with model providers including Anthropic, OpenAI, and Google to help translate Kotlin-specific insights into model improvements.
• Build open-source Kotlin benchmarks that can serve as a standard reference for the ecosystem.
• Create benchmark datasets that cover the full range of Kotlin usage, including server-side development with Spring and Ktor, multiplatform projects, Gradle-based builds, Android, library development, and more.
• Combine mined real-world tasks with carefully designed synthetic tasks that test specific Kotlin capabilities.
• Keep the benchmarks evolving so they remain challenging, relevant, and resistant to contamination as models improve.
• Take ownership of work from the first discovery of a problem in agent traces through evaluation design, experimentation, and delivery of fixes.

What the team is looking for

• Hands-on experience building evaluation or analysis pipelines for large language models or AI coding agents in either a research or production environment.
• At least three years of strong Python engineering experience, with the ability to write clean and maintainable code in data-intensive and ML-adjacent codebases.
• Experience analyzing data at scale, including querying large datasets with SQL or Athena, building data pipelines, and performing statistical analysis of experiment results.
• The ability to manage projects independently from identifying a problem in agent traces through to designing an evaluation, running experiments, and shipping a fix.
• A product-aware perspective that focuses on how developers actually use agents and how real failure modes should be turned into evaluation and training work.
• Comfort with Kotlin, or a strong willingness to become deeply proficient in Kotlin and work in Kotlin codebases every day.
• Strong fundamentals and enthusiasm are valued; candidates do not need to match every qualification exactly.
• The company says it is happy to provide training and support to help the right person grow into the role.

Additional experience that would be especially useful

• Experience with post-training LLM workflows, including SFT, RLHF, DPO, and GRPO, whether through direct training work or by designing the data and reward pipelines used in training.
• Familiarity with modern deep learning frameworks such as PyTorch and LLM training stacks such as TRL, verl, Megatron, or similar tools.
• Background in AI agent development, including tool-using agents, multi-step coding workflows, and agentic frameworks.
• Experience with evaluation tools and frameworks such as Inspect AI, Promptfoo, LM-evaluation-harness, or custom evaluation systems.
• Experience with experiment tracking and observability tools such as Weights & Biases, MLflow, Langfuse, or similar platforms.
• Knowledge of the Kotlin ecosystem, including Android, Gradle, KMP, Spring, and Ktor, along with an understanding of the workflows developers expect agents to support.
• Experience contributing to or maintaining open-source projects, especially benchmarks or evaluation tooling.

What JetBrains offers

The company highlights a strong base salary with pay that is intended to match skills and experience. The role also comes with a flexible work setup, allowing time either from home or from the office, plus the ability to work remotely from abroad for up to 30 days per year.

Other benefits include extra time off, medical insurance support for the employee and family, access to conferences and courses, language classes, relocation support, meal support on workdays, mental health services, a sports benefit such as an on-site gym or club stipend, and internal events for company-wide celebrations and team gatherings. Some benefits may differ depending on location.

Work location and application details

The role is available across several locations, including Amsterdam, Belgrade, Berlin, Limassol, London, Madrid, Munich, Prague, Warsaw, and Yerevan. The application form also includes a remote option for Germany and notes that candidates who are not based in one of the listed locations can discuss relocation possibilities with the team.

To apply, candidates submit an application through JetBrains’ Greenhouse form. The form asks for standard details such as name, email, phone number, resume, cover letter, LinkedIn profile, website or portfolio or GitHub link, how the candidate heard about the role, and preferred location.

Equal opportunity and privacy

JetBrains states that it is an equal opportunity employer and aims to create an inclusive workplace that welcomes people regardless of background, identity, religion, age, accessibility needs, or orientation. The company also notes that applicant data is processed according to its Recruitment Privacy Policy.

Degree Requirement: Not Specified

Visa Sponsorship May be