A Look Inside Our Bug Bounty Process

We believe in working together to build a secure platform for millions of merchants and customers worldwide. Our bug bounty program is crucial to this mission, and we couldn't do it without you—the talented security researchers—to help us stay ahead of potential vulnerabilities.

To be transparent, here’s a behind-the-scenes look at what happens when you submit a report, how we assess impact, and the care we take in resolving vulnerabilities. By sharing this process, we aim to build more trust between us and our hacker community.

Table of contents

What Happens When You Submit a Report (TL;DR)

 

  • Report Submission
    • A report is submitted and enters the "New" state, where it waits to be picked up by a triager.
  • Initial Triage
    • A triager performs an initial assessment and may close the report at this stage if it doesn't meet the eligibility criteria or scope. If it passes, the triager collaborates with the asset owners to confirm the validity and initial impact assessment.
  • Move to Triage
    • After verifying the bug, we work collaboratively with other security team members, product owners, and software engineers to assess the impact and determine what, if any, action we take as a result of the report.
  • Impact Assessment & Reward
    • At least once per week, an engineering panel meets to review triaged reports. The panel uses their diverse knowledge and experience to align on the most accurate score, ensuring a fair and thorough assessment.
    • After the panel has aligned, we assign the bounty according to that score. Our aim is to complete this step within one week of triage, but it may take longer.
  • After Triage
    • The report then awaits a fix from the engineering team. Once the fix is implemented, we may retest the issue ourselves, ask for your help, or both.
  • Resolution and Disclosure
    • If the fix passes testing, we resolve the issue and review it for possible disclosure.

Initial Triage

When you submit a report to our bug bounty program, it sets off a carefully structured process designed to ensure your findings get the attention they deserve. Here's what you can expect during the initial triage:

  • Gain Understanding: We carefully review the submission to ensure we understand the issue. If any details are unclear, we may ask for more information from you.
  • Core Eligibility Check: We verify that the report meets the core eligibility requirements as outlined in our policy.
  • Check for Duplicates: We search for any existing reports that may cover the same issue.
  • Determine Validity and Perform Initial Impact Assessment: If the report is valid, we perform an initial assessment of its potential impact.
Why Some Reports are Closed as 'N/A' or 'Informative'

As part of the initial triage, we might close reports as 'N/A' or 'Informative' based on the following criteria:

  • N/A (Not Applicable)
    • Reports are closed as N/A when they don't meet the basic criteria outlined in our policy. Common reasons for this include:
      • The report violates our policy or code of conduct.
      • The issue is related to assets that are outside the scope of our program.
      • The report describes a known issue or one specifically ineligible under our policy.
      • The issue is not reproducible or does not demonstrate a security vulnerability (e.g., functional bugs or non-security-related behavior).
  • Informative
    • Informative reports are those where we acknowledge the submission but take no further action. These reports often fall into the following categories:
      • Reports that reflect best practices or opinions—we often make deliberate choices that may differ from external expectations.
      • Reports of behaviors working as intended to facilitate other features.
      • Situations where an external researcher might reasonably perceive a security issue, but upon closer inspection, it presents no actual risk to Shopify.
      • Issues where we have accepted the risk to support certain functionalities or where compensating controls are already in place.

Move to Triage

After verifying the bug, we work collaboratively with other security team members, product owners, and software engineers to further assess the impact and determine if we're going to make changes as a result of the report.

Impact Assessment & Reward

After verifying the bug, we collaborate with security team members, product owners, and engineers to fully assess the impact. This ensures we consider different perspectives and expertise to understand the broader implications of the vulnerability. Our goal is to ensure transparency and fairness throughout the process, so you understand exactly how we arrived at the final score.

We use a bug bounty severity calculator, inspired by and mapped to CVSS scores, to generate a numerical score representing the impact of your report. This score maps to a specific dollar amount in our rewards table.

During our investigation, if we find that the impact is greater than what was proposed in your submission, we increase the score and explain why. If the impact is lower than initially believed, we adjust the score downward and provide an explanation.

Once aligned on the score, we'll award your report. We aim to do this within 7 days of triage.

After Triage

 

  • Critical and High-Priority Reports
    • Critical and high-severity vulnerabilities get immediate attention. Teams engage quickly to resolve these issues. In some cases, full triage and reward assessment may happen after the issue is fixed to prioritize safeguarding the platform. We'll keep you updated throughout the process.
  • Medium and Low Severity Reports
    • These reports are still important but often follow a more measured response, depending on the risk they pose. They might be scheduled for future sprints or handled as part of planned updates.

Note on Time to Fix

The time it takes to fix an issue should not be used as a reliable measure of how we perceive its impact. Some low-impact reports might be resolved quickly if they're simple and the team has time, while more complex issues, even with lower severity, may take longer.

Resolution & Disclosure

When a team ships a fix for an issue, it gets queued for retesting. We may do this ourselves, ask for your help, or both. Once the fix has been confirmed, we'll close the report as resolved and may consider it for disclosure at that time.



The Challenges We Face in Triage

Bug bounty triage isn't without its challenges. Here are a few common hurdles:

  • Complexity of Reports: Some bugs are straightforward to reproduce, while others require more investigation and collaboration across teams, which can slow down the process.
  • Volume of Submissions: High-priority issues are handled first, but all valid reports are important to us. Lower-severity reports might take longer to address, but they are still valued.

Be Patient and Trust the Process

We understand that waiting for your report to be assessed can be tough—especially when you're eager to see the results of your hard work. While we strive to handle each report quickly, some reports take longer to triage and resolve due to their complexity or volume.

Your patience during this time is appreciated, and we want to assure you that we're working carefully to give each report the attention it deserves. We value your contributions and want to ensure we get things right.

What Happens When We Get It Wrong

Even with the best processes in place, mistakes may happen. Here's what we do:

  • Let Us Know: If you believe we've misunderstood or made a mistake in handling your report, reach out and provide details about where you disagree. We'll re-evaluate the report, considering your input and any additional context you provide.
  • How We Re-Evaluate: We'll review the original report, your feedback, and re-open it for discussion with an engineering panel. If we find elements were handled incorrectly, we'll correct the report.
  • Correcting and Adjusting: If we find that we scored the report too low due to a misunderstanding, we'll adjust the score and award you the difference. Our goal is to be fair and transparent when we discover mistakes.

Final Thoughts

We deeply appreciate the contributions of the hacker community in making Shopify a safer platform. Your work helps us protect millions of businesses, and we're committed to handling your reports with fairness, transparency, and care.

Thank you for being a vital part of our security efforts. We look forward to continuing this journey together!

Feedback is a gift! We invite you to share your feedback about your experiences with our program—what's going well and any suggestions for improvement.