How to Compare Bug Bounty Solutions for Web3 Companies
When your web3 business must choose a bug bounty vendor, you need to list several decision-making criteria.
You can segment those criteria into 4 groups:
- Primary Expertise
- Customer Industries
- Top Customers
- Researcher count
- Triage team
- Review score
- Report Template
Experience & Compliance
- Date Established
Bug Bounty Solution Criteria Explained
When choosing a business solution, you must understand if they’ve worked with your digital asset types.
If not, then they may not have the right experience to solve your security challenges.
For example, if you have a desktop app that runs on smart contracts, you may need to check if the solution vendor has worked before with desktop apps that run on smart contracts.
The same applies to web & mobile apps as well as blockchain protocols.
Some software features are specific to a certain industry.
For example, Ethereum NFTs use smart contracts that run on the ERC-721 and ERC-1155 token standards. In contrast, Ethereum-based tokens leverage the ERC-20 standard. So if you are working with Ethereum NFTs, you would need to find out if the bug bounty solution providers have found vulnerabilities specifically in ERC-721 tokens.
Bug bounty solutions can have all sorts of customers.
However, it’s the crucial ones that provoke thoughts such as:
Well, if they chose them, it must have been a good decision.
Would they have invested their time in all that research just to pick a lousy solution?
Another way to determine whether a bug bounty program is efficient is by spotting great blockchain projects/products.
For example, if you work for a blockchain project, you’d look up to projects like Avalanche and Vechain.
If you work for a crypto exchange, you’d be interested in names like FTX and Huobi.
It’s also useful if you talk to people from these companies.
This way you can ask them if the bug bounty solution was a good fit.
Bug bounty solution pricing consists of 4 components:
- Reward budget
- Annual license to access the platform
- Annual triage
- Bug reward fee
A reward budget is a direct cost you need to pay to bounty hunters when they discover vulnerabilities.
Web3 cybersecurity is an emerging space. There is a huge demand for cybersecurity services and only a few companies can offer the right supply. That’s why the reward budgets for web3 bug bounties can easily range from $50 000 to $500 000.
An annual license to access the platform is a yearly subscription to connect with bounty hunters on the platform.
Annual triage is an annual cost of a support team that checks the vulnerability reports from researchers.
Some solutions bundle this together with the license.
A bug reward fee is a fee per valid bug that goes to the platform.
This criterion evaluation depends on what you expect from the bug bounty solution.
If you need many hackers, highlight the solutions that have the highest count of ethical hackers.
But keep in mind that quality is more important than quantity.
Suppose you were running a DEX project, what would you have chosen?
- 1 million whitehats who have not worked with DEXes, or
- 7000 whitehats who have found DEX vulnerabilities before
Note that this metric only shows the total number of signed-up users.
For every platform, there are only a fraction of all users who are active every day.
Triage teams manage bug reports.
They validate the raw reports coming from the researchers.
Triage teams are handy in situations when:
- You don’t have the time to manage the bug reports
- You need experts who can validate if a bug severity level is correct
- You would like to mediate the communication with researchers in case they turn rogue and try to hack your crypto wallet
Customer reviews demonstrate how the bug bounty solutions deliver on their promises.
But then, there are 2 issues with bug bounty reviews:
- The total score averages reviews from both businesses and bounty hunters
- The reviews can be fabricated or bought
We’ve researched up to 6 review websites for each solution, to solve the first issue.
We’ve then filtered the business reviews from the ones posted by ethical hackers.
The final score comes only from the business.
As for the second issue, we did our best to weed out the reviews from suspicious accounts.
Note that case studies and success stories are better than reviews.
This is because they can describe expectations and the results of working with a solution.
If your development workflow heavily depends on your software stack, you’d need to check the solutions that integrate with your software.
For example, if your developers use Jira, you could ease the bug fix flow by integrating the solution with Jira. Alternatively, if you use Telegram for work, you’d want to receive bug report notifications in your Telegram messenger app.
Having ready-to-use integrations speeds up the automation.
As a last resort, you can request access to API or webhooks.
Vulnerability reports should be structured to make sense.
If you have certain expectations for the report structure, check them out in our comparison tables below.
Experience & Compliance
For some people, a solution company must have been around for a while.
When a company has an established presence in the market for a good long time, it shows 2 things:
- They’ve learned how to grow their operations
- They’ve fleshed out their workflows
Working with established companies implies fewer mistakes in your work interactions.
Make sure to align your solution with your compliance policies. But you don’t have to be too strict.
Remember that policies dictate how the vendors store data about you, not your customers.
HackerOne vs Synack: Industry-Asset Match
|Mergers & Acquisitions||–||–|
|Top Customers||US Department of Defense|
Navy Federal Credit Union
Bank of Georgia
HackerOne vs Synack: Competing Criteria
|Annual License||$22 000||$38 000|
|Annual Triage||$50 000||Included|
|Bug Reward Fee||X||X|
|Researcher Count||1 000 000||1 500|
HackerOne vs Synack: Workflow Differences
|AWS Security Hub||+|
HackerOne vs Synack: Experience & Compliance
|UK Cyber Essentials||+|
Compare Top Bug Bounty Solutions for Web3 Business
There are more bug bounty solutions to review than just HackerOne and Synack.
We’ve prepared a business guide that compares 6 bug bounty solutions for blockchain companies:
Click here to get this guide and you’ll save yourself at least 50 hours of research.