Sveriges mest populära poddar
Arrested DevOps
Arrested DevOps

Something About Security With Ben Hughes

56 min•9 september 2014

Om avsnittet

Security, the Etsy Way

Ben Hughes of Etsy says he's one of the senior network engineers there, though his team looks after infrastructure, about a thousand machines and a hundred laptops, and leaves networking to a NetOps team. He fell into security after dropping out of school when he discovered 2600 and Linux, and at one point had around 1,500 accounts on his high school's Unix system. He came to Etsy from Puppet Labs, where he learned about DevOps Days.

His current work includes Python on Midas, a host-based intrusion detection system for the Mac that Etsy released with Facebook, and Go to trick Logstash into using different TLS ciphers. Next week he's at an incident response conference, because for breaches "the days of it being an if are long gone."

Blameless Phishing

Matty recalls Ben's point about phishing: the question is what you do once someone gets phished. Ben says Etsy focuses on people first. You can't shout at a recruiter for opening a PDF, since that's their job, and "blameless postmortems work for blameless phishing campaigns." What matters is that people tell the security team when something screwy happens, and "the more you chastise people, the less they will tell you." So when someone reports a phish, they say thank you and give them an Etsy gift card. Being able to respond matters more than believing you have full coverage: "Nothing's 100% in security, despite what some people will try and sell you."

Scary Clouds and the Trusted Network That Isn't

Matty raises the belief that you can secure your own stuff better than a cloud provider. Ben says that if you bring the "armadillo" security model of the last 20 or 30 years into the cloud, "you're going to have a bad time," because unless you build a private network, there's no trusted internal network at all. Matty passes on what a Microsoft data center manager once told him, that after the US government Microsoft is the most attacked entity on the internet, so who better to learn from.

Ben's caution is about outsourcing security to someone else. Without a complete overview of your organization and your threat model, "you're going to be hit by a surprise," and security vendors sell silver bullets using fear. Etsy is big enough to build its own tools, as Netflix and Square do for their particular problems, and those tools "you can't buy commercially."

Don't Throw It Over the Wall to Security

Ben says throwing something over the wall to security at the last minute is as bad as throwing it over to operations, and everyone who's had a bad time says they should have talked to security sooner. If he has budget to allocate, he'd spend it on a security team early, not on products or pen testing, because until you map what you're defending and from whom, you can't build defenses. Etsy's first security person was one of its toolsmiths who took on security on the side, and he says you can find people interested in security in operations, development or QA.

For developers, his number one request is "please stop turning off TLS verification," and the age-old rule of never trusting user input: "never trust users," in the nicest possible way.

Proxies and the People Who Route Around Them

Matty rants about transparent HTTPS proxies that decrypt traffic, which break a gem install because the certificate doesn't verify, so people turn verification off. Ben gives the benefit of the doubt, since there are good reasons to inspect web traffic, and suggests trusting the proxy's CA cert in the gem config. He says people will route around anything that stops them doing their job, with SSH tunnels, OpenVPN or tunneling IP over DNS: "However you need to get out of a network, you will." So "we can either work with these people or kind of be avoided by these people."

Trust famously doesn't scale, he says: at 50 people you know everyone's name, and at 10,000 you don't trust other buildings. Etsy defaults to trusting everyone, monitors everything, and is transparent about what the security team is doing, which is how you get trust back. He sums up DevOps as "why don't we work together?"

Secrets, Passwords, and Breaking Things on Purpose

For a listener's question on secrets, Ben says Chef encrypted data bags "turn your encryption problem into a key management problem," and points to Nordstrom's Chef Vault, which Matty says ships with ChefDK. He'd also like Square to release its resident-only in-memory file systems built on FUSE. His main advice is to get to where you can change a secret and nothing breaks: once sharing works, randomly change some passwords, "it'll break a ton of stuff, and you'll have a terrible day," but you'll know next time credentials leak, which they do. "Pastebin is a treasure trove of other people's mistakes, and GitHub Gists is just a shopping cart full of logins."

Personally, he uses a password manager with around 600 credentials and two-factor on everything. Matty asks front-end developers to name password fields password so managers can find them, and mentions his six-word Diceware passphrase, which he learned by making 1Password prompt him every time for a couple of days.

What People Get Wrong

Ben's first misconception is that HTTPS is too slow, and he points to istlsfastyet.com. His second is the media's fixation on zero days, which "99.99999% of organizations do not need to worry about," when the unpatched Linux kernel and users' four-character passwords are bigger problems.

Trevor asks how to start security where there's no team. Ben says find the person who stays up late on IRC, give them time, and get leadership to treat security like backups. Some security is better than none, and "smaller incremental gains is always the way to do it." To sell it, he says you can do it "not out of fear, but out of this is the responsible thing to do," and customers pick the vendor that's more secure. Matty realizes there's no padlock in a mobile app to tell you whether traffic is encrypted, and Ben says he'd like platforms to show one only when certificates are validated. He tells of a game site that showed a picture of a padlock next to an HTTP URL.

The Villain in The Phoenix Project

Matty asks whether InfoSec is still the right name, given the band Information Society, and Ben says it's still used, that his own LinkedIn said Security Monkey for a long time, and that hacker versus cracker is settled. On The Phoenix Project's security manager, who's cast as a villain, Ben says many see security as Big Brother, and that Etsy's team held a hackers party watching the film Hackers, taught people to pick locks, and ran a security hack week. If you're the villain, he says, "start being nicer to people because you work with them."

He also thinks security needs to buck up its own ideas: the military jargon, like the phrase threat intelligence, which "brings me out in a rage," makes the field seem aggressive. He wants less preaching that it's 100% or nothing, noting that even CPUs ship with errata lists, so "computers are broken from the ground upwards" and it doesn't all have to be doom and gloom.

  • What exactly do you security folks do all day?
  • So let's talk about ZOMG SCARY CLOUDS
  • I'm a developer. What do I need to know to help me be pals with InfoSec?
  • Ops people know security, right? Or not?
  • Common security mistakes/misconceptions
  • How can InfoSec be better buddies with other functions?
  • Are you tired of Matt calling it "InfoSec"? Does it remind you of Information Society?

Check Outs

Matt

Ben

Trevor

Fler avsnitt

Visa alla avsnitt av Arrested DevOps

Arrested DevOps med Matt Stratton, Trevor Hess, Jessica Kerr, and Bridget Kromhout finns tillgänglig på flera plattformar. Informationen på denna sida kommer från offentliga podd-flöden.