Interview with Kevin Dubourg, Bug Bounty Program Manager, Yousign

Why did you decide to go for such a new and disruptive solution as Bug Bounty?

There are a number of platforms out there, which – mostly US based. We asked for certain guarantees on the hunters invited to our programs, and it seemed to us that YesWeHack offered those guarantees and the confidence to launch a Bug Bounty program.

What value you think Bug Bounty can add compared to traditional cyber security solutions (e.g. pen test)?

Kevin Dubourg, Bug Bounty Program Manager, Yousign

Diversity in terms of perspectives and skills. Every hunter has his own approach, his way of doing thing, a unique approach that makes a particular attack. This is different from pentesting, and it provides a much stiffer challenge. With Bug Bounty, we kind of left behind the pentest world, in order to benefit from 10, 20 or 30 different views and really challenge our teams.

What is really interesting, is that not all hunters are necessarily “cyber security professionals”. The entire ecosystem is represented here, and we can pick up individuals based on their nationality, skill set, ranking on the platform, etc.

Yet Bug Bounty’s main value, is the continuity, recurrence and “annualization” of the tests: as soon as we release a new version, we integrate the existing program and get immediate feedback on the new version’s security level. We don’t need to wait a year for the next pentest to check on the security of our update. This approach is embedded within our project lifecycle. Our scope evolve constantly, and bugs evolve at the same time. Security flaws turn up every day, not just once a year, and Bug Bounty enables us to detect and fix them in time. It helps us monitor our services on a practically constant basis, and that is very reassuring. It would also be impossible financially to do an pentest on each delivery, although we would really need to, etc.

And then there is ROI. Yousign carries out one pentest each year. And this is quite expensive, compared to a Bug Bounty program. It’s a bit crazy when you think about it: they cost more or less the same, but Bug Bounty covers an entire year, whereas an audit only lasts a week…

Is Bug Bounty the end of pen testing? Or will it always remain complementary?

For Yousign, it will continue to be complementary. It could mean the end of the pentest in some industries, but not in ours: as a trusted third-party provider we are required to carry out regular audits. In a less stringent regulatory environment, I would probably consider about using Bug Bounty only.

However Bug Bounty is very important for our sales and marketing : it’s clearly a differentiator to large prospect accounts. We mention it systematically in our request for proposal submissions as it’s seen by the market as a quality hallmark.

What are the differences between the results of a pentest and Bug Bounty?

I’ve had the same reports from both, but there are clearly many more reports from Bug Bounty than from pentest. And after having carried out a pentest on a given scope, running Bug Bounty always brings additional vulnerabilities. One of the problems with pentests is results mainly depend on the expertise of the pentester. Our last pentest showed up some relevant things, but when you compare its results with those of the Bug Bounty program we launched afterwards… there’s no comparison.

Have you seen any changes in your teams since you have been using Bug Bounty?

Of course. To start with, I managed the programs on my own, then fairly soon afterwards I got the development teams involved so that they could directly reply to hunters, fix bugs, etc. Most reports concerned the applications team, so they had to face up reality and take things forward, if I can put it like that.

What’s more, we quickly saw that their interactions with the hunters affected their delivery and working methods: not only do they integrate security into their development work more effectively, they actually started to “think” differently, always keeping in mind security aspects. You could say that they are not only delivering for clients, but for the hunters too (laughs).

What’s next?

The next step is to use Bug Bounty even more. In addition to the current programs on our production and “staging” environments, we want to fully embed Bug bounty within our CI/CD workflow to add to our battery of functional and unit test. This should make us even more agile and bug bounty a key component of our CI/CD approach.

And later on, we might move to a public program.

Case study of a Trust Service Provider (TSP) on private Bug Bounty program

What made you decide to launch a Bug Bounty program? 

We mainly launched a bug bounty because of our short delivery cycles. We were used to doing “traditional” pentests once a year, but as we have a lot of changes every month on our scopes, we simply could not wait 12 months for the next audit. Bug Bounty enables us to carry out continuous checks, for each release, update, new delivery, etc.

What value can Bug Bounty add compared to traditional cyber security solutions (e.g. pen testing)?

ROI: being able to pay for results only is very important for a small organisation like ours with limited budgets. With traditional pentests, we have to pay even if nothing if nothing has been found. Our last pentest cost around €8,000 and no major vulnerabilities were reported. 

After two months running our program, dozens of security flaws had been reported, including some critical vulnerabilities never reported through previous audits, for a reward budget totalling around half of the cost of a single audit. 

I would also mention diversity – pen testing is too “academic” and just don’t meet our real needs. Most pentesters run tools and tick boxes: as a result there are too many things, too many vulnerabilities, that aren’t found. The diversity of hunters and their range of skillset make a big difference.

Lastly, the model is super flexible. In terms of scope evolution for example: with a traditional pentest, scope is defined in advance – if you want to change anything, you have to pay again for another audit. Now, with Bug Bounty, I can fine-tune the program over time, I can add products or URLs to the scope – which is key to us.

Is Bug Bounty the end of pen testing? Or will it always remain complementary? 

As a trusted digital service provider, we have to run pentests to meet regulatory requirements. So, we have no choice but to continue doing traditional audits. However, if we were in an industry not subject to such regulations, there’s no question we would only use Bug Bounty. 

This year, we are going to mention bug bounty in our certification process, making the case that Bug Bounty is equivalent to intrusion testing – and actually more effective.

What’s next? 

Expanding the program to our APIs and mobile apps. 

Is there anything else you’d like to mention? 

Bug Bounty is also a key selling point for our sales team – especially with large accounts that require the most stringent security guarantees. Bug Bounty is now automatically included in our sales presentations to large accounts. 

Case Study: Groupe ADP’s Public Bug Bounty Program

What made you decide to launch the Bug Bounty program? 

Daniel Diez Head of the Digital Factory Division, Groupe ADP

Daniel Diez – Head of the Digital Factory Division, Groupe ADP :
“The Group Security team took the lead on this project. I had no prior experience of Bug Bounty, but we very quickly saw the model’s advantages and power. And although I never had any particular doubts or worries, now all I see is the benefits. The bugs reported by hunters are vulnerabilities that we and our auditors may not have seen otherwise, and which could therefore be exploited by bad guys.” 

Eric Vautier – Groupe CISO, Group ADP :  In cyber security, anticipation is everything. You need to stay a step ahead of the hackers. This means keeping a close eye on market innovations.

In the digital world, doing “old-style” protection means you are clearly behind the game. And a good way to catch up is to work directly with security researchers who use hackers’ methods and think like them.”

Daniel Diez: “We started Bug Bounty wondering if we could successfully adapt the model to our way. Today, it is one of the pillars of our web security strategy. Of course, it’s vital to set the program’s rules carefully: you have to structure the tests in the right way so that the hunters don’t “disperse” their efforts. You need to identify the right “boundaries”, and this is where the program setup is essential. We started with a tightly drawn scope and expanded it as we went along.”

What value can Bug Bounty add compared to traditional cyber security solutions (e.g. pen test)?

Daniel Diez : “Continuous testing – this is Bug Bounty’s big strength. Pen tests are run at a given point in time – not following every minor delivery. While with Bug Bounty, we have hunters working continuously, remaining alert to anything new, which means they can detect whether any change creates potential vulnerabilities.”

Eric Vautier: “In a perfect world we should systematically test each update on our website. This would mean running a pentest every week, or even more often… And everyone knows that’s not feasible. Bug Bounty makes such continuous verification possible.”

Daniel Diez: “I’ve been getting reports we never got from our pen tests – way more in-depth reports, particularly on the website navigation experience. Auditors don’t necesserily have this approach. What’s more, a pentest has a limited timeframe, whereas hunters take the time they need to go as far as possible. As time goes by, they also get increasingly familiar with our scope, which means they can go even more in depth.”

Eric Vautier:  “A Bug Bounty program can also be used to report more functional, not just technical, application vulnerabilities. For me, this is what genuinely differentiates it from the pentest. It is a completely different angle. A pen test often relies on automated tools, while Bug Bounty builds on these tools with a more human approach.”

Daniel Diez : “What is also interesting is the interactions with hunters. They help us understand the vulnerabilities they’ve found and how to fix them effectively. In this way we can leverage their expertise. 
For sure, Bug Bounty demands some investment. You have to be available to understand what the hunters have tried to do, to talk to them…

But they force us to ask ourselves fresh questions: How a bad guy would get round our protection measures?”

Is Bug Bounty the end of pen testing? Or will it always remain complementary?

Daniel Diez: “For me, neither works without the other. For one thing, we are not necessarily testing the same things with both. And we cannot set off into the unknown without having some minimum level of certainty in advance. Bug Bounty comes in at a more mature stage in the logical flow of events. You need to leverage a minimum base level of security before launching Bug Bounty. That said, today, within the current scope, we no longer need to run pentests. Bug Bounty is enough on its own. You need to set the bar at the right level at the outset, and it then becomes a recurring process.”

Eric Vautier, Group CISO, Groupe ADP

How does Bug Bounty fit with your agile approach?

Daniel Diez: “Like everyone, we have tools to manage sources, builds, projects and performance analytics. We also use tools to log and track each new vulnerability report from the Bug Bounty program. For each sprint we verify which relevant data we can include, so we can deal with issues as they come.”

Why have you gone public? How has that changed your approach with Bug Bounty?

Eric Vautier: “The main advantage is to maximise our risk coverage by multiplying the number of potential tests. Also, it gives us a single channel for reporting vulnerabilities in our website.”

What comes next?

Eric Vautier: “We are going to open up new scopes, on other applications and with other business entities using the same model: private program first, then going public.”

YesWeHack provides its bug bounty platform and expertise to the French Armed Forces Ministry.

YesWeHack is delighted to support the French Cyber Defence Command (COMCYBER), in order to leverage its 3,400 cyber-combatants+ force.

YesWeHack, a French start-up and bug bounty leader in Europe, equips COMCYBER with an innovative concept and tool to boost cooperation with all the Ministry’s cyber entities.

This bold initiative is part of the Ministry opening up towards the civil society and private actors.

Florence Parly, the French Armed Forces Minister, announced on the 22nd of January :

A partnership has been established between COMCYBER and a start-up, YesWeHack. So, yes, I do announce: we will launch the first bug bounty of the French Armed Forces Ministry at the end of February 2019. Ethical hackers, recruited within the cyber operational reserve, will be able to search for vulnerabilities in our systems and, if successful, be as they should be, rewarded.

Florence Parly, the French Armed Forces Minister

With the signing of this partnership, the Armed Forces Ministry becomes the first French Ministry to launch a bug bounty program. COMCYBER will leverage YesWeHack bug bounty platform to meet the growing challenge posed by new cyber threats.

With the YesWehack bug bounty platform, COMCYBER will be able to best use its trusted community of reservists, in order to improve global security of the ministry’s entities

Guillaume Vassault-Houlière, YESWEHACK CEO

This bug bounty program opens new perspectives for the management of the operational cyber reserve. Ultimately, such initiative will make possible to train reservists and increase their skills to significantly and durably improve the Ministry’s level of security.

[ITW] High value bugs : like the hunters, these are the bugs we find most exciting !

Quentin Berdugo CISO @dailymotion

Can you describe dailymotion and the role you have within the organization?

Since 2005, dailymotion has been pioneering video streaming and delivery and is now making its comeback as a major video destination platform. I’m dailymotion’s CISO.

What is dailymotion’s history in terms of coordinated vulnerability disclosure and what milestones have you been through?

When we saw our first user notification *on Facebook*, we realized that we were lacking a proper channel for our users and the security community to notify us of potential issues.

For our users, we created a security category on our support portal, with instructions for the support team as to how to route these specific inquiries. For the security researchers, we had a address created.

This went a long way and we had some surprisingly interesting notifications from the users, the InfoSec community and academia.

Since we later introduced a private bug bounty program, we were able to use it to reward these spontaneous notifications.

This didn’t really prevent the occasional researcher from tweeting about an issue before they even gave us a head’s up, but it really helped us build a strong experience on vulnerability disclosure that turned out to be very useful when writing our disclosure policy, that we published at the same time as we opened the bug bounty to the public.

We have made this disclosure policy available in our “security.txt” file, an draft internet standard aiming at facilitating the disclosure of security issues.

You have recently opened up your bug bounty program to the public, what’s your feedback? + Read More

“Ein Bug Bounty Programm ist eine gute Möglichkeit, um die eigene Arbeit auf den Prüfstand zu stellen”, bekräftigt Yves Berquin, Mitbegründer von MatrixReq.

Bitte stellen Sie Matrix Requirements und Ihre Rolle im Unternehmen kurz vor

Bevor wir 2014 Matrix Requirements ( gründeten, waren wir Projektmanager bei einem Medizintechnikunternehmen und hatten erkannt, dass wir für die Rückverfolgbarkeit des Designs ein besseres Tool benötigten. Daher entwickelten wir MatrixALM zunächst für den Eigenbedarf.

Die Gründung von Matrix Requirements zur unabhängigen Vermarktung dieser Anwendung erfolgte erst später.

Matrix Requirements ist ein vierköpfiges Team, das bereits 100 Kunden mit insgesamt 700 Nutzern akquiriert hat, was für ein so kleines Team eine beachtliche Leistung darstellt.
30% unserer Kunden kommen aus den USA und ähnlich viele aus Deutschland, der Rest entfällt auf die übrigen europäischen Länder sowie Israel, Australien, Indien und Kanada.
Meine Aufgabe im Team bezieht sich vorwiegend auf Back-Office, Netzwerke, Datenbanken und Linux-Server. Es versteht sich von selbst, dass Sicherheit bei mir höchste Priorität hat.

Was hat Sie dazu bewogen eine Bug-Bounty-Übung anzusetzen?

Auch wenn wir ein kleines Unternehmen sind, haben wir die ISO13485:2016 Zertifizierung erhalten und streben auch die Zertifizierung nach ISO27001 an. Diese Standards erfordern die eingehende Untersuchung der mit unseren Prozessen verbundenen Risiken. Ein offensichtliches Risiko in Unternehmen wie dem unseren ist natürlich das unbefugte Eindringen Fremder in unsere IT-Systeme. + Read More

“A bug bounty program is a practical way to put your work to the test” states Yves Berquin – CoFounder of MatrixReq

Yves Berquin - Cofounder of MatrixReq

Yves Berquin, Cofounder of MatrixReq – GmbH

Presentation of Matrix Requirements and your position

Before we co-founded our German company, Matrix Requirements ( in 2014, we were project managers in a medical devices company and it was clear to us that we needed a better tool to manage the traceability of the design. We built MatrixALM for ourselves and later on we created Matrix Requirements to launch our application independently.

Matrix Requirements team is 4 people which is quite honorable compared to our results so far: we have about 100 customers totaling about 700 users.

30% of our customers come from the US, about 30% from Germany and the remaining in rest of Europe, Israel, Australia, India, Canada.

My role in the team is more on the back-office, network, databases, Linux servers. Needless to say I’m very concerned about security.

What are the reasons that led you to embark in the bug bounty exercise ?

Even though we are quite small, we are certified ISO13485:2016 and on the way to be ISO27001, and this type of standards mandate that we study the risks of our processes. Of course one obvious risk in our type of business is the intrusion of our information systems.

We’ve had intrusion attempts in the past an we protected ourselves with quite elaborated active rules on our firewalls. We’ve had an audit from a group in KULeuven, and one of their recommendations was to go through a bug bounty exercise.

Why did you chose YesWeHack ?

We first asked a well known US bug bounty company but the pricing was out of reach for us. Then we discovered YesWeHack, through the OVH DLP accelerator (we are also members). We contacted them and found out quickly that their offer matched what we were looking for: a group of researchers that could investigate our security in BlackBox mode. In particular we wanted to be able to talk to the researchers in English and that is a given on that platform.

What are the results of your private phase ?

The private phase was achieved with a group of 10 researchers, and they came back with 5 vulnerabilities. Frankly, we were relieved that none of the reported vulnerabilities were severe, which confirmed that we already had quite a good security maturity.

Of course we can never rest in this field, but what were returned to us were subtle weaknesses that wouldn’t allow by themselves anyone to actually enter our site.

We rewarded the researchers anyway, understanding that sometime a combination of small weaknesses could lead to an actual attack vector. The exchange with the researchers were very fruitful and they gladly checked that our fixes were efficient as well.

That dialogue is really the positive aspect of the exercise: we forced ourselves to reply quickly to the remarks, and they were very quick to answer back and offer suggestions to solve the issues if needed.

What are you waiting from the public phase ?

Opening the bounty to all the ethical hackers on the platforms in YesWeHack should lead to much more return for us, and should help us solidify even more our application and its API. I hope nothing too bad will come out of it but of course I prefer hearing about it this way: we have to detect potential security issues as soon as possible.

A bug bounty program is a practical way to put your work to the test. We hope to learn a lot from this public phase – through ways that we wouldn’t have thought about ourselves.

Today more than ever (think Facebook, British Airways, …) we must stay humble and remember that ‘Security through obscurity’ doesn’t exist and it’s only by putting your cards on the table and be pro-active that you can ensure a decent level of security.


Go to MatrixAlm’s Bug Bounty Public Program !


OVH Bug Bounty RetEx by Vincent Malguy

As OVH bug bounty manager from March 2016 to March 2018, Vincent Malguy, through this interview, delivers his return of experience to share some tips with people who wonder how to set up and manage a program.


The genesis

In the early 2010’s, many companies in the IT sector like Facebook or Google started to launch bug bounty programs and within OVH this appeared as an obvious need. However, it took time to frame the project and to meet all the operational conditions to take the leap.

In 2015, when I was recruited by OVH, it was time to put in place all the bricks to calmly launch a bug bounty.

Back in the day, we identified two issues: the issue of vulnerability export and the legal complexity when paying rewards.

Of course, we evaluated the possibility of launching it without external help but we quickly gave up the idea because it is not our core business.

In any case since the beginning, it has been clear in our minds that a real bug bounty program is, in the long run, a program open to a wide audience.

In January 2016, we met with Korben and Freeman. They presented YesWeWack’s roadmap to launch the first European bug bounty platform.

The timing was perfect and we decided together to launch OVH’s public program on the occasion of “la Nuit du Hack” in June 2016.

Private phase

In this exercise we have the support of the management and technical teams.

Based on that internal mobilization, we started to carry out an additional audit on the initial scope in order to ensure its maturity. We then worked with the communications, legal and accounting teams. Once these prerequisites were gathered and validated, with YesWeHack, we started with a 1 month private window.

+ Read More

Bug Bounty: Take the leap – [ITW] Alain Tiemblo @BlaBlaCar

Alain Tiemblo – BlaBlaCar Web Security Lead Engineer

Since September 2017, BlaBlaCar has been managing with a select number of security experts a private Bug Bounty program to enhance the operational security of its platform.

Previously accessible only by invitation via, YesWeHack’s bug bounty platform, this program has enabled BlaBlaCar to remain proactive on the cyber security of its services.

Thursday April 19, BlaBlaCar’s program is public

What is your role at BlaBlaCar?

I am a backend developer profile, today overseeing application security. When I joined BlaBlaCar, I was in charge of the platform’s performance and security. In mid-2015 and early 2016, our operational security needed to level up significantly, especially following our major fund-raising campaigns, which put BlaBlaCar under the light and pressure. So at that period of time, i took the lead of a small team to mitigate these attacks, and audit/consolidate the platform.

What is your approach to security, including coordinated vulnerability disclosure?

We have kept application security in-house for a long time. Previously, we used classical audits conducted by various companies, by several basic pentest applications, by using static analysis tools, etc. I think it helped to rough out a lot of little things that would have been detected by bug hunters.

In addition, we received a few troll messages on Twitter reporting vulnerabilities without notice and without any details… We also have some emails via customer support about potential security holes, but nothing was disclosed by these contacts, they first wanted to be paid and this, without proof of the existence of a security flaw, so it was impossible for us to enter the game. + Read More

Bug Bounty : Franchir le pas – ITW d’Alain Tiemblo @ BlaBlaCar

Alain Tiemblo - Bug Bounty - Vulnerability Coordination

Depuis septembre 2017, BlaBlaCar propose à un nombre d’experts en sécurité triés sur le volet, un
programme de Bug Bounty privé afin de renforcer la sécurité opérationnelle de sa plateforme. Accessible
jusqu’alors uniquement sur invitation via, la plateforme de bug bounty de YesWeHack, ce programme a permis à BlaBlaCar de rester proactif sur la cybersécurité de ses services.

Entretien avec Alain Tiemblo Web Security Lead Engineer – @BlaBlaCar – manager du programme de Bug Bounty.

Jeudi 19 avril le programme de BlaBlaCar est public

Quel est votre rôle au sein de BlaBlaCar ?

Je suis un profil développeur backend, aujourd’hui chapeautant la sécurité applicative. Lorsque je suis arrivé à BlaBlaCar, je m’occupais de la performance et la sécurité de la plateforme. Mi 2015 début 2016, nos besoins en sécurité opérationnelle ont augmenté de manière significative notamment à la suite de nos grosses levées de fonds qui ont suscité quelques convoitises. J’ai alors pris le lead d’une petite équipe afin de mitiger ces attaques, et auditer / consolider la plateforme.

Quelle est votre démarche en termes de sécurité et notamment de divulgation coordonnée de vulnérabilités?

Nous avons pendant longtemps gardé la sécurité applicative en interne. Auparavant, nous faisions appel à des audits classiques menés par diverses entreprises, par plusieurs applications de pentest, en utilisant des outils d’analyses statiques, etc. Je pense que ça a permis de dégrossir beaucoup de petites choses qui auraient été détectées par des chasseurs de failles.

Par ailleurs, on a reçu quelques messages de trolls sur Twitter signalant des failles sans préavis et sans aucun détails… + Read More

1 2