Trust and Safety Engineer
We’re hiring a Trust & Safety Engineer to analyze threat actor behavior, detect emerging abuse patterns across large datasets, and design production-ready mitigations that keep the platform safe without slowing developers.
This role blends security engineering, data analysis, and applied LLM techniques, partnering closely with Product, Infrastructure, and Security to turn evolving threats into scalable defenses.
This is a remote role and may be seated anywhere in the US, however significant overlap with core hours in the Pacific time zone may be required.
Responsibilities include, but are not limited to:
- Analyze threat actor behavior and evolving abuse patterns across large, heterogeneous datasets (livestreams, chat, follows, gifts/coins, reports, device/account signals) to inform detection logic, policy decisions, and mitigations.
- Design and develop production-ready detection and enforcement systems that disrupt abusive behavior in real time (risk scoring, rate limits, automated actions, human-in-the-loop review, ban evasion controls).
- Build and maintain internal moderation tools: case management, evidence capture, appeals, and audit trails to optimize for speed, consistency, and operator wellbeing.
- Research, prototype, and implement LLM-driven techniques for abuse detection, classification, and prevention (e.g., policy-aware triage, semantic clustering of new attack patterns, reviewer assist, automated labeling with evaluation gates).
- Integrate ML/moderation classification systems (e.g., computer vision, text moderation, speech/audio signals; vendors such as Hive / Rekognition-like services / OpenAI) into your existing moderation structures with strong safeguards (thresholding, fallbacks, debuggability, measurement)
- Design and maintain backend services that power Trust & Safety workflows (APIs, streaming/near-real-time pipelines, event logs, idempotent enforcement).
- Own data systems: SQL-backed models and analytics, plus caching/queues like Redis for fast, consistent tooling and enforcement decisions.
- Operationalize policy into systems (including transparency/accountability considerations like CA AB 587 reporting-style rigor where relevant).
- 4+ years software engineering experience (or equivalent) building production systems.
- Full-stack experience with React + TypeScript (internal tools / ops UIs strongly preferred).
- Backend engineering experience: designing APIs/services, workflow/state machines, and production integrations.
- Experience with databases (SQL) and caching/queues such as Redis.
- Demonstrated ability to work with large datasets to identify patterns, ship mitigations, and measure impact.
- Ability to integrate new detection products into existing moderation workflows (queues, triage, reviewer ops, appeals, auditing).
- Prior Trust & Safety / abuse prevention / content moderation or security engineering experience at a UGC or live platform.
- Experience working with moderation classification or ML systems (CV/NLP/audio), vendor tooling, and production rollout patterns.
- Applied LLM experience: evaluation pipelines, red-teaming/adversarial testing, drift monitoring, and safe automation.
- Familiarity with policy enforcement concepts (graduated enforcement, appeals, auditability, transparency-style metrics/reporting).
This role has starting wage/salary range of $129,000. The full range for this level and function within FiveBy is $129,000 - $202,000. These ranges are applicable for the labor market of the Greater Seattle Area. It is common for FiveBy to start employees below the midpoint of the range. Final base salary is directly related to each candidates' unique qualifications and experience.
FiveBy employees are eligible to participate in FiveBy’s group medical, dental, and vision insurance plans. FiveBy covers 100% of monthly premiums for individual employees – additional qualifying beneficiaries may be added at the expense of the sponsoring employee. FiveBy also provides 401k with employer matching, 120 hours of PTO annually (pro-rata for hourly employees), and 10 paid holidays per year.
FiveBy is an equal opportunity employer.
All employment decisions – including the decision to hire, promote, discipline, or discharge – are based on merit, competence, performance, and business needs. We do not discriminate based on race, color, religion, marital status, age, national origin, ancestry, disability (as defined by applicable law), medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, veteran status, or any other status protected under federal, state, or local law.
Work environment and physical demands
In order to perform this role, the selected employee must be able to operate within a standard office environment, with or without reasonable accommodation. This includes the ability to withstand long periods of staring at a computer screen, exposure to fluorescent lights, and repetitive motion associated with writing and/or continuous computer use. This role requires the ability to stand or sit for long periods of time, communicate with others via speech or text, and use hands or other tools to operate a computer and keyboard. Light to moderate lifting may be required. Due to the nature of our work, employees must be able to uphold the stress of traveling to client sites as well as adapt (within reason) to the conditions of client sites. Regular, predictable attendance is required, as is the ability to work outside of standard business hours as business needs dictate.
Accommodation
If you require accommodation due to a documented disability covered under the Americans Disability Act, click here to submit your accommodation request. To be eligible for accommodation under the ADA, applicants must meet the minimum qualifications of the position for which they are applying.