devopsdays Chicago 2019

Matty and Trevor record live at devopsdays Chicago 2019, the sixth time the event has run, with three speakers from the program. Jessie Frazelle, who is self-employed, talked about open source firmware and roots of trust and why they matter. Veronica Hanus, a contractor in Brooklyn, gave an Ignite talk on how developers' attitudes toward code comments affect growing developers and, later, their documentation styles. Jeff Smith, Director of Production Operations at Centro, talked about why ethics in technology is difficult. It is Veronica's first devopsdays ever and Jessie's first Chicago. Jeff says it's the fifth time at devopsdays Chicago, and that an open space on ethics the year before helped inform the talk: "a conversation that needs to continue to happen and grow."

First Impressions and Fifth Ones

Veronica arrived wondering how public transit works outside New York, and found a team that helped with transportation, served "the best food you've ever eaten at a conference," and even asked speakers what song they wanted when walking on stage. Jeff says the event has grown and the audio and video have gotten smoother, and calls it "the bar at which I judge all of the other conferences." Jessie praises how well organized it is and the varied audience, including a data scientist and a technical writer, as a way to avoid groupthink. Veronica's favorite conversations were in open spaces about bringing the lived human experience into a talk submission alongside the technical content.

What Could Go Wrong

Matty sees a parallel between Jessie's talk and Jeff's, since it may have seemed a good idea to put a web server somewhere it didn't belong, and asks how these things can be misused. Jeff says we misapply technology ourselves in operations and engineering, choosing tools because they're interesting without thinking through downstream consequences. The example is the recent Zoom breach, where a local web server was meant to make reinstalling the software easier. Jeff recalls Google's image-classification mistake involving Black people and says "there wasn't a Black guy in that room," so the fix is broader perspectives and thinking one step beyond the task in front of you. Trevor proposes asking "what could go wrong" seriously instead of facetiously, and Jeff says an IAM policy statement ID Jeff saw in AWS was literally "what could go wrong," with allow star star.

Jessie says many employees are process-oriented, so if asking the question isn't in the process, you do A, B, C, D and finish the task. Veronica says it comes down to psychological safety, and the innovation possible where it's okay to ask why something is the way it is differs greatly between organizations. Trevor ties it to Jeff's morning idea of an ethical body for technology: people feel pressure to finish but rarely the moral pressure to challenge getting it done. Jeff adds that standards and culture move the bar, recalling a job 10 years earlier where a password in a config file wasn't crazy, until a later job where the reaction was "what's wrong with you?" Jessie says it's culture: if the culture is to make money even at the expense of exploiting customers and nobody faces ramifications, "that's the culture that you set."

Veronica remembers a government research center that wouldn't put anything in the cloud, where losing data was an expected part of the process: "You lose data, you grieve a little, and then you put your head down and create it again." Looking back, Veronica says, it sounds bizarre.

Building Psychological Safety as a Leader

Matty asks Jeff, as someone who leads teams, what to do. Jeff's first answer is vulnerability and being unafraid to say "I don't know." On Jeff's team that's an okay answer, just not the end of the conversation. Jeff also pushes education as part of the job, asking why people who are already on call should learn a new technology on their own time, and says spending an hour or two a week reading in the kitchen is fine. The third is creating space to dissent constructively, so people who disagree know "we're still a team."

Veronica says the difference between helpful and unhelpful workplaces was time to learn and how things went wrong. In a research job, a machine had a problem nobody noticed for a month, and the team traced it back in what Veronica calls a blameless way, and everyone left knowing how the machine worked inside and out. Veronica says both fields produced the same pattern: learn at work instead of being burned out in the evenings, and when something goes wrong, focus on the process more than the person who made the commit.

Influence Without the Title

For an individual contributor without that safety, Jessie says candor with leadership can go one of two ways, depending on the person. Jeff says to think of leadership as a role and not a position, and that whoever is the emotional leader of the team can provide some psychological safety cover, such as going to the manager to say something isn't working, or inviting a quiet teammate who is stewing over a technology to speak. Veronica adds "praise in public, reprimand in private," and to remember a manager may feel just as vulnerable, so pick a moment instead of ambushing someone in the hallway.

Matty brings up a talk by Anwen Simmons a couple of years earlier on lending privilege, and says one mark of a team with a high level of psychological safety is that everyone speaks about equally. Jeff says that as experience and confidence grew, calling out a bad idea got easier, and that if fired tomorrow Jeff is confident of getting another job, so using that to shield someone else can be huge. Jessie agrees, having started out unable to be aggressive and get away with it. Trevor adds that lending privilege doesn't have to happen in front of the person: talk about their idea in conversation and remind people whose idea the team is now following.

Key Messages and Plugs

Jeff's one message is that we should take responsibility for what we create and own the consequences, because the stakes get higher as technology advances. Veronica's is "Comments are a form of documentation, and documentation is good." Jessie's is that people working on different layers of software should talk to each other more and actively listen, to solve problems between the interfaces.

Jeff is writing a book tentatively called Real World DevOps, written from the perspective of an individual contributor in imperfect organizations, due out in 2020 sometime. Veronica announces, for the first time publicly, the Quick Developer Guides YouTube channel with two other first-time speakers, a series of 10-minute videos on topics from getting started with version control to budgeting for a career change into development. Jessie, who is not actively looking for a job, plugs two books, Soul of a New Machine and Ben Horowitz's The Hard Thing About Hard Things. Jeff says, "She's not looking. She's choosing."

Matty and Trevor chat with guests Jessie Frazelle, Veronica Hanus, and Jeff Smith in front of a live studio audience at devopsdays Chicago 2019.

Community & Event Stuff

If you have an upcoming conference you would like to see promoted on ADO, you can fill out the handy form at arresteddevops.com/conf

Upcoming conferences

Open CFPs

• lots of DevOpsDays

Discount codes

• ADO2019 for 20% off lots of devopsdays
• Velocity Berlin Nov 4-7 2019 - discount code "ADO2019" gives 20% off for Gold, Silver, and Bronze passes; early price price ends 20 September.