Know your community – Berend-Jan Wever (SkyLined / @berendjanwever)

Aspiring ASCII artist, a chef, a gardener, bug bounty hunter and one of the leading browsers vulnerability researchers. Please meet Berend-Jan Wever AKA SkyLined!

Questions

Q: How many years have you been working in the security field?
A: Probably about 30 years. My first experience in security was as a kid, when my computer got infected by the Tequila virus in the late 80s. I reverse engineered it and started creating my own viruses.

I wanted to see if I could bypass existing antivirus software and escape detection by being very stealthy. Obviously I never released anything into the wild, unfortunately I never published any of this and lost the floppy disk on which I stored all my work, so it’s gone forever.

My next work in security was when I started using the internet in the late 90s. I started looking for, and finding, XSS vulnerabilities in pretty much all websites I looked at. But what really drew me in were the CodeRed and Nimda worm outbreaks of the early 2000s.

Here was something very similar to the viruses I worked on in the past, but with a much more efficient way of spreading from machine to machine. I started reading up on security flaws, looking for vulnerabilities in software and writing exploits for public vulnerabilities.

I created a website to publish my work, and started finding vulnerabilities in Microsoft Internet Explorer in the process. The exploits I published for these MS IE vulnerabilities in particular led to me getting a job in security at Microsoft.

Q: What was your motivation for getting into the security field in the first place?
A: The biggest motivation is the thrill of being able to do things that you are not supposed to, especially if you can do it on a very large scale and/or without anyone noticing. It gives you a sense of power unlike anything else. I also get a huge satisfaction from preventing anyone from abusing such issues for their own personal gain by getting them fixed.

Q: What was the first vulnerability you found?
A: It was a buffer overrun in the SETI@home client. Users would install this software on their machine to help NASA’s SETI project process the huge amount of data that they collect. The client would download and process data from the SETI servers when the machine was idle, and send the results back to the server.

I found that the code that handled HTTP responses did not check the length of a HTTP header before writing it to static stack based buffer. An attacker on the local network could man-in-the-middle the connection to the server and modify server responses to trigger this vulnerability and execute arbitrary code on the machine.

SETI had a statistics page on their site that showed this software was installed on over 100,000 machines, most of which where at large enterprises. It appeared that the server was vulnerable as well, so theoretically, this vulnerability could be used to take over the server and from there all these clients.

Q: How did you feel when you found the vulnerability?
A: I was really excited that I finally found something as I had been looking for vulnerabilities in various software for a while already. Also, the scale and wide spread of the vulnerability was quite large. It was a bit frustrating that I was not able to write an exploit for the Windows version of the client though, because I did not know Windows internals or exploitation techniques at that time.

Q: Did someone help you?
A: I was lucky enough to be on a private email list called “0-day digest”, which had a lot of really smart people on it. Some of the people I respect most in the security community were on this list. We exchanged 0-days and exploits, and helped each other with writing them. I found it tremendously useful and I learned a lot from them. One of them created an exploit for the BSD version of the SETI client for instance.

Q: What is your field of expertise in vulnerability research?
A: For over a decade, I’ve been heavily focused on low-level memory safety issues in web-browsers. Over these years, I’ve created more and more sophisticated fuzzers and analyzers to automate the process of finding issues and analyzing them to determine if they have security implications.

It has been my goal to create a system that can automatically look for, find and report vulnerabilities in *any* software, so that I don’t have to do anything other than spend the money I made off vulnerability reward programs.

Unfortunately, my progress in this direction has, so far, kept pace with the improvements in browser security and my cost of living, so the amount of work I do has not decreased significantly.

Q: Is there some security research field that you always wanted to learn but never had a chance?
A: I would love to dig into many different subjects, but Windows kernel has been on the top of that list for years now. It makes even more sense nowadays, when you really do need to escape the sandbox after you successfully run code with a browser bug.

Q: Where do you work at today?
A: For the past five years, I’ve lived of vulnerability reward programs and the occasional freelance project. I work mainly from home, which has the benefit of allowing me to see my family a lot. But it does mean I can get isolated from the community if I don’t actively engage them. Therefore I’m currently trying to do more projects, because I want to look at new things and meet new people.

So far that’s working well for me; I’ve made some very useful professional contacts. But more importantly, I’ve had a lot of fun on various very different projects and made new friends. So, if anyone has a fun project that they would want to work on with me, do reach out!

Q: What would be your dream job? pure research? exploit development? relaxing at the beach?
A: I’m very lucky in that this to me is a well paid hobby, so I really do not have a dream job. The only way I could improve on what I have is if I could get more revenue out of less work. That would allow me to invest more time into things that do not pay off immediately, such as learning about new topics, or work on high-payoff but also high-risk projects.

My long term goal is to get to a point where I can quickly set up very effective automated testing for pretty much any software, and have this testing produce results that make sense to the vendor. If people no longer need me, but only my software, I can do whatever I feel like all the time, instead of most of the time like now.

I saw in your blog that you focus on browsers vulnerabilities. You found and reported:

• 8 vulnerabilities in Microsoft Edge
• 30 vulnerabilities in Microsoft Internet Explorer
• 6 vulnerabilities in Google Chrome
• 2 vulnerabilities in Firefox
• 1 vulnerability in Opera

Most of the vulnerabilities are Memory related vulnerabilities – memory corruption / use-after-free / read data out-of-bounds (information disclosure)

Q: What methodology do you use to find these kind of vulnerabilities?
A: I have been developing my own fuzzers for browsers for over a decade. I probably wrote over a hundred of them over the years, most of which were small fuzzers purpose built to target a particular feature. But I also created tens of generic fuzzers that I ran or am still running in my own framework on up to 100 VMs.

Some fuzzers take only a few hours to make, some have been in development for years, as I add more features to expand their coverage, or remove old code for features that are no longer supported in modern browsers. Most of my fuzzers get a practical name, such as HTTPFuzzer, RegExpFuzzer, DOMFuzzer, JSFuzzer, etc.

If you want to live off bug bounties, you have to make sure you find different vulnerabilities than the rest of the community. Creating your own fuzzers is a good way of doing that in my experience. I have considered getting out of fuzzing and selling my existing fuzzers.

I asked around and found a few parties that were interested. However, I realized I would not have any control over what anyone did with the issues they would find, nor if they shared the code with others. The same would be true if I open-sourced it. This would surely lead to bugs found with my code being used for purposes I would not approve of, which is not acceptable.

Realistically, I could only sell or give them to the browser vendors themselves. However, the price would have to be substantial enough to offset the loss of income I can expect when they are using they same fuzzers I am. I don’t expect anyone has budget for that unfortunately.

Q: What is the setup for the fuzzers?
A: I have around 8-10 servers, each one of which runs anywhere between 4 and 20 VMs, depending on the hardware specs. These are just very fast but cheap desktop machines and they tend to die on me quite often, as they are not designed to be run continuously at 100%. As a result the numbers can vary, but I’m running somewhere between 50 and 100 VMs at any given time.

Each VM runs a copy of the framework, which works independently from the others. All of them store their results on a central shared disk, which I can browse to see what they found. It’s pretty basic, but it works well enough for me.

For crash detection and analysis, I use BugId. This is a Python script I wrote that wraps around a debugger and detects exceptions in the application that could indicate a bug was found. It analyses the exception and the application state in order to determine the type of bug that triggered the exception, and whether it may be a security issue or not.

I normally tell my framework to ignore any crashes that BugId thinks are not obvious security issues, as its false negative rate is very close to zero. I’ve open-sourced BugId, so if you want to see how it works and give it a try yourself, you can get a copy at https://github.com/SkyLined/BugId.

Once a security issue is found, the framework tries to reproduce the issue by feeding the same data my fuzzer generated to the application again and seeing if it crashes in the same way. If it does, than it tries to reduce the size of the data by cutting out chunks and seeing if the remainder still reproduces the issue. This is a crude but effective way of removing everything not involved in triggering the crash and reducing the size of the repro to something manageable.

At this point I manually go through the data to create a copy that does exactly what is needed to trigger the crash and only that. From this minimized repro and the report created by BugId, I can often tell what the problem is. I may sometimes do a little reverse engineering to find out more, or attempt to write a PoC exploit, but most often I will just send what I have at this point of to a vulnerability reward program.

Q: Which parts in the browser do you find most intriguing? (Sandbox / JavaScript VM / JS -> callback ->native)?
A: I like the network protocols, the DOM and Javascript, as you can tell from the types of bugs I found.

Q: What is your favorite method to exploit the vulnerabilities?
A: I used to write exploits when it was easy enough, but nowadays it takes weeks to write a working exploit because most bugs are not very straightforward and because of all the mitigations you need to bypass. Since vulnerability reward programs do not give you nearly enough extra money for a working exploit to be cover your time, I’ve stopped doing it.

Q: Which browser are you using?
A: I use both Firefox and Chromium.

Q: In your opinion, which of the common browsers is the most secured?
A: That depends on context and I could not give you a satisfactory answer in a few lines. I believe they are all pretty close and getting closer every day thanks to the hard work of all vendors.

You have worked with several Bug Bounty programs:

• ZDI
• Exodus Intelligence
• iDefense

Q: Is it possible to make a living from bug hunting?
A: Yes: I have been living mainly of bug bounties for the last 5 years.

Q: Did you ever report vulnerability to a vendor and got a hostile response?
A: Yes, unfortunately vendors still come up with excuses not to spend time and money on address security issues asap. Some will say they are too small to devote resources, or that their release process is too complex to push security fixes through faster.

Unfortunately, I feel that if they leave their customers vulnerable because they are incapable or unwilling to do better, these customers should know this, so they can make an informed decision about potentially switching to a similar product from another vendor. Needless to say, these vendors do not want me to do that.

Q: What is the longest period of time it took for a vendor to patch a vulnerability you reported?
A: I honest did not keep track, but it probably was over a year. However, since I introduced a 60-day deadline on all my reports directly to vendors, which I will only extend if the vendor gives me a good reason, no vendor has taken that long :).

Q: What was the silliest reward you got for reporting vulnerability to a vendor?
A: Microsoft has given me Microsoft branded clothing for not releasing 0-day, after they told me I was not eligible to ever participate in their Bug Bounty because I insist that they fix their bugs within a certain time-frame. I found it quite odd that they though I would actually want to wear it.

Q: Do you think the vendors made progress in the past few years of how they handle vulnerabilities reports from the security community?
A: Definitely, but the progress is very slow, and it requires a lot of pushing from the community.

Q: Do you think the rewards security researchers get from report vulnerabilities are fair?
A: No. I think vendors are putting their customers at risk because they are not spending nearly enough on security. It is a calculated risk; they optimize profit and they do not carry the real impact of a vulnerability; that is the customers’ problem. The vendor only risks losing a few customers who rightly attribute their problems to the vendor’s lack of investment in security, assuming they have a reasonable alternative. The spent more on marketing to change perception about this attribution, than they do on fixing the problem in the first place.

Q: Have you noticed a lot more non-traditional companies and organizations showing interest in bug bounty programs?
A: Yes, as more and more of our lives happen online, and more and more of our machines are connected, more and more companies will be affected by computer security issues – even those not directly involved in online services.

Q: What industries or business sectors would you like to see more involved in the bug bounty business?
A: All of them; the impact of security breaches to companies is getting bigger. I think it’s big enough to be worth investing in preventing it from happening the first time, rather than preventing it from happening again.

Q: What are the best companies to work with when hunting for vulnerabilities? What traits do they have in common?
A: There are those that will help you and answer questions, and those that will fight you and anything in between. I’ve always enjoyed finding vulnerabilities when the vendor is actively trying to frustrate me, but it’s also nice when you can have a good discussion with the vendor and help them improve their security even more that way.

Financially, finding bugs in Microsoft software has been most rewarding for me, because it is pretty much everywhere, so vulnerability reward programs are very interested.

Q: You are a very experienced researcher and you had the opportunity to participate in many security conferences. What is you favorite security conference?
A: I don’t have a personal favorite, but I do like smaller technical conferences because they tend to have a higher ratio of smart technical people.

Q: What kind lectures you like to attend? listen to?
A: Technical stuff, anything that may contain some cool tricks that I might be able to use, or ideas that I can apply to other topics.

Q: How do you choose your lecture topics?
A: I like to talk about things that I would like to listen to myself. So, I don’t pick topics because they are particularly useful, but because I think it would be nice to hear about. A good example of such a topic was my talk on ASCII art shellcode; completely useless in every day live, but I found it to be worth knowing that it’s possible.

That doesn’t mean it’s always useless: I’m currently doing talks on BugId, which is a project I am working on that allows you to automatically detect and analyze bugs in applications. I think a lot of people would use it if they knew it existed and what it does, so I figured people would want to hear me talk about it.

Q: What do you love most in conferences? (conference events – CTF / hacking village / Hack the badge, drinking parties etc)
A: I don’t do any hacking at conferences themselves; I do enough of that at home already. For me conferences are all about talking to my friends in the security community and meeting new people. I do attend a few of the most interesting talks, but most of the time I like to hang around the lobby chatting.

Q: In your opinion, how did the international security community change in the past 5 years?
A: I’ve seen a continuation of a longer term trend in that a lot more money is going in to computer security. Unfortunately, this seems to attract people who like money more than it attracts people who like computer security, and the later are not as good at attracting the money as the former.

As a result, there are more bad apples peddling snake oil, pretending to help defend the end-user, but not really improving security. They take money out of the security community while at the same time ruining its reputation.

Also, money going into attack seems to grow faster than money going into defense. Today, you can make a lot more money in developing exploits for malware or government agencies than you can in working with vendors to make the end-user more secure. It must be hard for many people not to water down their ethics when they can make a very, very large amount of money by simply sending a working exploit to a vulnerability broker, rather than to the vendor.

The fact that governments are actively funding this is sending a wrong message to young people coming into the community, and I expect we’ll have to deal with the consequences of that in the future.

Q: What type of products do you like most looking into vulnerabilities in?
A: Servers that handle complex protocols. Unfortunately, I haven’t done that in ages, but nothing beats a wormable vuln.

When I first met you, I noticed you have on your left arm a smiley tattoo. Why did you tattoo it? Is there a story behind it?
A: Yes, I do have a smiley tattoo on my arm. I’ve been very happy with my life for a long time, and wanted to get something that would remind me of that. I first got a very small smiley tattoo while I was in Australia in 2003. I wanted something that wasn’t too in-your-face. Then about two years ago I lived next to a tattoo shop and one day while I was hanging out over there, I told them I was considering getting a larger smiley tattoo and asked them for ideas.

They knew I was moving house a few months later, so they offered to get me a larger smiley for free as a going away present. One of the eyes of the current smiley is where the original used to be. The original reminded me of the great time I had when I was young and single, the new large one adds all the great memories I have made since then in my old home, where my first two kids where born. Maybe in another 10 or so years, I will have to get an even bigger one. 🙂

Q: What’s the single most important piece of advice you would want to give for someone seeking out a career in the security filed?
A: Publish as much as you can, even that really hacky code that you’re embarrassed to show. Everybody was a n00b once, everybody writes crappy code at some point, and nobody is an expert at everything.

Publishing allows others to find you and get a good impression of your skills and your progress over time. Over time, you will build up a name for your self and a reputation. I have no qualifications; I did not even finish University. Pretty much my entire career is based on people knowing me and what I do, and that is all because I’ve published it online.

Q: What are your hobbies?
A: Behind the keyboard, I like to create ASCII art and write Javascript demos, particularly 1k, though I haven’t done the later in a looong time.

AFK I like playing football, cooking, gardening, home improvement, and dinner and drinks with friends. I like going to Ibiza together with my wife; mostly we just drive around the island, do some shopping and eat at all these fantastic places. I like to go swimming with my kids and we play with Lego a lot.

It was a pleasure, Berend-Jan, to talk to you

You’re welcome.