PAI Program Initiative Research Fellow: ABOUT ML
The Partnership on AI (PAI) is seeking a Research Fellow to join the high-growth, high-potential ABOUT ML initiative. This Research Fellow will report to the lead for ABOUT ML and will conduct multistakeholder research with PAI Partners to support the development of evidence-based recommendations for documentation of machine learning systems. Specifically, this fellow will begin by identifying the current challenges for implementing documentation practices within ML development and deployment teams through a mixture of qualitative and quantitative research methods. The Fellowship is a paid, salaried position and will begin Summer 2020, lasting 12 months.
Background
The ABOUT ML initiative aims to set a new industry norm of creating useful and accessible documentation for all machine learning systems. Documentation for machine learning has been identified as a foundational and powerful enabler of transparency and accountability for machine learning and AI, making it possible for more organizations to produce high quality and useful documentation on their ML systems serves PAI’s mission of AI benefitting people and society. Many PAI Partners are excited about the prospect for ABOUT ML’s recommendations to contribute to growing responsible practices within AI development and deployment.
To ground the ABOUT ML recommendations in previous research, expertise, and experience from other domains that deploy documentation (e.g. healthcare, financial services, consumer products, chemical safety, etc.) as well as existing work on ML documentation specifically, ABOUT ML contains the following multistakeholder input processes:
Public Comment
Steering Committee
Diverse Voices
Solicited Expert Input
ABOUT ML aims to produce evidence-backed, effective recommendations for how organizations can develop the tooling and processes to create documentation for the machine learning systems they build as well as research-backed templates for how best to present the documented information for different audiences. In order to achieve these goals, PAI will be conducting several large scale multistakeholder research projects and pilots with Partner organizations and beyond.
About this role
This Research Fellow for ABOUT ML will report to the lead of the ABOUT ML project and conduct crucial research to support the development of ABOUT ML recommendations and pilot programs to test those recommendations with Partners. The first research project will be to investigate the challenges of implementing documentation for machine learning within organizations that build and deploy ML systems. These challenges may be behavioral (e.g. people find it frustrating to document their work), technological (e.g. documentation tasks are not well integrated into existing workflows), organizational (e.g. time spent on documentation is valued by managers), or of another nature. This initial research will set the foundation for ABOUT ML pilots, which will be designed and iterated upon to address these challenges. Additional potential research topics include: what is the relative difficulty of answering various documentation questions, what are best practices for phrasing documentation questions, what are best practices for presenting and adding new documentation questions into existing workflows, what formats of templates work best for different stakeholders, depending on the research interests and expertise of the fellow. The ideal candidate would have deep experience applying qualitative and quantitative research methods to questions at the intersection of machine learning, science and technology studies, HCI, user experience research, design research, and/or software development processes, especially within multidisciplinary settings.
This Research Fellow will lead on research planning, methodology, and execution, and work closely with the rest of the ABOUT ML team, internal, and external stakeholders to conduct this research. Possible activities may include contextual interviews with machine learning engineers, workflow analysis, survey design and execution, and ecosystem and stakeholder mapping. As this Research Fellow will be joining an ongoing initiative that encompasses research, design, testing, iteration, and implementation phases, the research questions may evolve over time as new needs arise or priorities shift within the initiative.
PAI is headquartered in San Francisco, with a global membership base and scope but we will consider remote work
Key responsibilities include:
Refine and plan a practical and robust research methodology to investigate the challenges of implementing documentation in machine learning
Execute the research plan in a manner that leverages the Partnership’s voice and perspective, rather than one company, lab, or organization’s
Draft and edit academic papers, white papers, and/or reports
Work closely with the ABOUT ML team to manage multistakeholder aspects of research projects
Translate research findings into relevant protocols and recommendations for ABOUT ML, adapting content for different stakeholders
Contribute to other ongoing research within the ABOUT ML initiative
Work collaboratively and independently with other PAI teams and external stakeholders
Attend industry events, convenings, and conferences
Key qualifications include:
Master’s or PhD (preferred) in HCI, Software Engineering, Industrial and Organizational Psychology, Sociology, Anthropology, or other related disciplines
Experience working in or other familiarity with software development or ML development/deployment environments
Excellent written and oral communication skills
Expertise in qualitative (e.g. interview-based) research, quantitative research, and user-centered design methodologies
Ability to lead project management aspects of research across numerous internal and external stakeholders
Demonstrated track record of or interest in cross-disciplinary engagement
Comfort navigating diverse input from a wide array of contributors
Ability to translate research insights into recommendations for practical implementation
Bonus points if:
You follow design justice principles https://designjustice.org/read-the-principles
You understand the ways in which software design is not neutral, and interacts with capitalism
You are experienced in how to change organizational culture and structure to incorporate societal concerns
PAI is a fast-paced organization that often requires quick turnarounds, so initiative and efficiency is critical for success in this position. Because of the varied and dynamic nature of PAI’s multistakeholder research projects, the ideal candidate should have experience working on projects with diverse and stakeholders.
PAI is proud to be an equal opportunity employer. We celebrate diversity and we are committed to creating an inclusive environment in all aspects of employment, including recruiting, hiring, promoting, training, education assistance, social and recreational programs, compensation, benefits, transfers, discipline, and all privileges and conditions of employment. Employment decisions at PAI are based on business needs, job requirements, and individual qualifications.
PAI will consider for employment qualified applicants with criminal histories, in a manner consistent with the San Francisco Fair Chance Ordinance or similar laws.
The Partnership on AI may become subject to certain governmental record keeping and reporting requirements for the administration of civil rights laws and regulations. We also track diversity in our workforce for the purpose of improving over time. In order to comply with these goals, the Partnership on AI invites employees to voluntarily self-identify their gender and race/ethnicity. Submission of this information is voluntary and refusal to provide it will not jeopardize or adversely affect employment or any consideration you may receive for employment or advancement. The information obtained will be kept confidential.
Application Materials:
1. Resume and/or CV
2. Cover Letter
3. 3-5 page writing sample (please append to cover letter): this can be any writing for which you were primarily an author and does not need to be about any topic related to this project or PAI
4. 3 References [name, title, email, phone]
Applications for this job posting will be accepted until July 15, 2020.