PCE Evals Lead

A role serving as a key horizontal driver on evals efforts across teams – leading sprints, devising and executing automation strategies for manual efforts, and coordinating resources with PMs. This leader will be expected to take on highest priority work and flex as we continue to transform the team and uplevel the AI capabilities at all levels. Meta’s Product Content Engineering team is responsible for the content experiences across platforms and products. PCE works in close partnership with engineering, product, and partnerships teams to improve the discovery and distribution of content to drive value for users at scale. The work is time-sensitive and innovative, situated at the intersection of content expertise and Meta’s industry-leading ranking and delivery systems. The main function of a content engineer is to define and execute a content strategy that aims to significantly improve the product experience for consumers. The ideal Content Engineer has technical expertise combined with a capacity to drive change in ambiguous, fast-paced and high growth environments. They will be expected to perform data analytical work, comprehend complex recommendation systems, and test hypotheses while driving alignment across a wide cross-functional team to deliver business impact at scale.

PCE Evals Lead Responsibilities

• Develop and execute strategies for FoA product teams to evaluate LLM models at scale across content work;
• Lead sprints for LLM model evaluations and auto judge creation in core PCE areas, while devising and implementing systems to increase evals capacity in areas involving the building of AI creator tools, content quality, content understanding and Search;
• Understand technical product needs and translate those into actionable direction for content-focused cross-functional teams
• Understand Meta’s strategic and competitive position and deliver content strategies and product solutions that are aligned with our mission and recognized best in the industry
• Define emergent opportunities and drive impact via execution in highly ambiguous areas. Agile support and flexibility is required

Minimum Qualifications

• 8+ years of demonstrated experience driving relevance on content platforms, with experience with video creators, ranking systems, content strategy, content labeling and analysis
• 1+ years working with GenAI products, prompt engineering, annotation, and/or content labeling and analysis
• Expertise in developing and implementing high-quality datasets, annotations, and evaluation metrics that drive the training and improvement of generation models
• Experience working independently and autonomously
• Previous experience collaborating with product and technical teams
• Proven knowledge of industry trends and news
• Experience leading and collaborating on cross-functional projects to build products, respond to urgent needs, preempt issues and address concerns in collaborative and timely fashion
• The Content Engineer should be experienced in content production and understanding, strategy and analysis to execute across a variety of projects, ideally in multiple languages

Preferred Qualifications

• 5+ years of work experience in product management, data analytics, or content operations in a leading content platform
• 2+ years of experience working closely with ranking teams
• Experience in data visualization and tools such as Tableau

Meta builds technologies that help people connect, find communities, and grow businesses. When Facebook launched in 2004, it changed the way people connect. Apps like Messenger, Instagram and WhatsApp further empowered billions around the world. Now, Meta is moving beyond 2D screens toward immersive experiences like augmented and virtual reality to help build the next evolution in social technology. People who choose to build their careers by building with us at Meta help shape a future that will take us beyond what digital connection makes possible today—beyond the constraints of screens, the limits of distance, and even the rules of physics.

For those who live in or expect to work from California if hired for this position, please click here for additional information.

$242,000/year to $296,000/year + bonus + equity + benefits

Individual compensation is determined by skills, qualifications, experience, and location. Compensation details listed in this posting reflect the base hourly rate, monthly rate, or annual salary only, and do not include bonus, equity or sales incentives, if applicable. In addition to base compensation, Meta offers benefits. Learn more about benefits at Meta.