The Internet Archive is seeking a Software Engineer to join the expanding Decentralized Web team. Since 2015, the Internet Archive has been a leader in applying decentralized technologies to digital preservation, democratizing access to knowledge, and building the open web. See some of our early experiments building a decentralized archive.org here. For more background on our DWeb work watch this staff talk by our Decentralized Web Tech Lead, Arkadiy Kukarkin.

DWeb projects include:

  • Processing, placement and retrieval of large (100TB+ to multi-PB) Archive collection datasets on incentivized storage networks like Filecoin and Storj to increase resiliency and global distribution of the material

  • Development of user-facing tools for viewing Archive materials stored on these networks

  • Preservation of Web 3.0-native digital works

  • Supporting independent website deployment on the decentralized web

  • Stewardship of humanistic, democratic, and open principles in the DWeb ecosystem

In growing the team, we seek to scale up this decentralized tech work to production level, and make IA items preserved in decentralized storage available to the patrons and the broader community.

The ideal candidate is an engineering generalist who is able to work across systems and outside of existing frameworks, integrating available tools and filling in the blanks as needed. The full scope of responsibilities includes automating and scaling data pipelines, administering and monitoring processes across multiple machines, and building tools and interfaces for both internal/administrative and end users. However, depending on your focus and breadth, subsets of this work can be flexibly allocated within the team.

Some prior familiarity/hands on experience with decentralized technology concepts is strongly encouraged, however deep protocol level understanding is not required for this role. Conversely, this position is an excellent opportunity to grow your understanding of the decentralized web field.

The Decentralized Web Engineer will be a part of a larger DWeb group of engineers, policy and communications staffers and community volunteers.

Essential Job Duties:

  • Develop tooling, dashboards, and automation to productionize experimental deployments of decentralized technologies within a complex network environment

  • Ensure continued uptime and performance of these systems

  • Collect observations on performance, behavior, and feature completeness of these technologies for communication to the protocol developers

  • Build user-facing interfaces presenting material from decentralized networks

  • Ensure non-interference with core Internet Archive technologies and resources and contain any disruptions

  • Support further experimentation with new technologies, benchmarking, and evaluation

  • Support further growth of the Decentralization team

Qualifications and skills:

  • Significant experience & commitment to system integration and tool-building

  • Working familiarity with decentralized technologies like IPFS/Filecoin, Dat, Secure Scuttlebutt, Storj

  • Some hands-on experience with IPFS (go-ipfs, js-ipfs), Filecoin (lotus, estuary) or other above-mentioned tools

  • Protocol-level understanding and debugging experience with any of the above a plus but not required

  • Strong skills in JavaScript (if front-end leaning) and/or Python (if backend leaning)

  • Working knowledge & familiarity with Linux system administration, Docker, monit/systemd/supervisord, etc

  • Experience with data pipelines and distributed computation a plus

  • Strong capacity for self-directed learning, operating in small teams, and technical communication

  • Ability to collaborate closely with people with different backgrounds and varying levels of technical expertise

  • Must demonstrate care and respect toward coworkers, outside partners, members of the public with whom this role interacts

Also a plus:

  • Participation in the DWeb community, meetup groups, and events

  • Participation in open source projects

  • Facility in Go and Rust

Job Details:

This is a remote-first position working in a distributed team. Candidates will need to have some time overlap with a primarily North America (including Pacific Time) based distributed team for collaborative work and meetings. The role reports to the Decentralized Web Tech Lead.

Benefits & Perks:

The Internet Archive is a remote first workplace and provides a comprehensive benefits package including; PTO, paid holidays, and medical benefits. Depending on where you live, we also provide these additional benefits; dental, vision, health savings accounts, flex spending accounts, commuter benefits, short term disability, long term disability and retirement programs.

At the Internet Archive, we believe we do our best work when our employees bring together diverse ideas. Members of all groups under represented in the tech industry and library world are strongly encouraged to apply. We are proud to be an equal opportunity workplace and are committed to equal employment opportunity regardless of race, color, religion, national origin, age, sex, marital status, ancestry, physical or mental disability, genetic information, veteran status, gender identity or expression, sexual orientation, or any other characteristic protected by applicable federal, state or local law.



This position has been filled. Would you like to see our other open positions?