Data Scientist Responsible AI
- Plaats
- Utrecht, Utrecht
- Opdrachtgever
- Coöperatieve Rabobank U.A.
- Gezien bij de bron
- vandaag 18:01
Kies je bemiddelaar
YER
Detachering| Tarief van het bureau | onbekend |
| Fee | onbekend |
| Reageren tot | onbekend |
| Transparantie |
Omschrijving
Build reliable GenAI solutions through robust evaluation, statistical validation and responsible AI guardrails.
De positie
As a Data Scientist in our Responsible GenAI team, you build the evaluation capabilities and guardrails that help teams across Rabobank understand, measure and improve the quality and safety of their GenAI solutions. You work at the intersection of data science, software engineering and responsible AI, translating real-world use cases and risks into robust evaluation methods, reusable tools and practical controls. Your work spans assistants, report generation, document processing and emerging agentic applications; not just chatbots.
This is a hands-on development role that requires strong statistical judgement and production experience. You design sound experiments, critically analyse data and metrics, and understand which conclusions the evidence does—and does not—support. You turn experiments into maintainable, tested, production-ready code and work with engineers to integrate it into deployed solutions. This is not a notebook-only role, nor does it require you to be an ML or platform engineer. The focus is not on training foundation models or conducting standalone research, but on making evaluation and safeguards work in practice.
You help answer:
“How do we know a GenAI solution performs as intended—and how do we test that its evaluations and guardrails are reliable?”
You assess existing approaches and new techniques critically, reuse suitable components where possible, and develop new methods where they add clear value.
Your Responsibilities
- Build reusable evaluation frameworks and tools that support repeatable testing, comparison and regression analysis across GenAI use cases.
- Develop and critically validate evaluation metrics, combining deterministic checks, model-based evaluation and human assessments. Assess metric reliability, uncertainty and whether metrics measure what they are intended to measure; make clear which conclusions the data supports and where it falls short.
- Design statistically sound experiments, test datasets and evaluation scenarios, including sampling strategies, representative data, ground-truth data, edge cases and challenging inputs, together with use-case teams and domain experts. Investigate how dataset quality and representativeness affect findings.
- Develop and assess guardrails using appropriate methods, including rule-based controls, LLM-as-a-judge approaches and trained ML classifiers, for risks such as sensitive-data exposure, unsupported outputs, prompt injection and unintended system behaviour. Select approaches based on the use case and evidence, and train or evaluate classifiers where appropriate.
- Measure guardrail effectiveness and trade-offs, including precision and recall, detection thresholds, class imbalance, missed detections, false positives, robustness, drift, latency and cost.
- Deliver maintainable, tested, production-ready code and work closely with engineers to integrate evaluations and controls into CI/CD, APIs, pipelines, deployment, monitoring and production workflows, with reproducible results and clear diagnostics.
- Translate use-case needs into actionable evaluation criteria, explain findings and limitations, and turn delivery experience into reusable methods, documentation and tooling.
Who Will Thrive Here
You enjoy turning the question “Can we trust this result?” into a concrete test, a measurable criterion and a useful tool.
You are curious about why AI systems—and their evaluators—behave the way they do. You combine statistical rigour with a builder’s mindset: you enjoy designing methods, critically analysing data and metrics, and turning experiments into…
… lees de volledige omschrijving bij YER.