User Guide

SaMD Complaint Risk Assessment · Jira Cloud app

Quick start

This app adds an AI-powered risk assessment panel to any Jira issue. It is designed for medical device quality teams who need a fast, consistent first-pass triage of complaints — classifying whether an issue is a product defect and scoring its probability and severity against your configured risk matrix.

Prerequisites

Jira Cloud (the app does not work on Jira Server or Data Center)
An Anthropic API key — create one at console.anthropic.com
The app installed from the Atlassian Marketplace

First-time setup checklist

Install the app from the Atlassian Marketplace.
Open any Jira issue and click "View app actions" (below the issue title).
Select "SaMD Complaint Risk Assessment" to open the panel.
Click Settings (top right of the panel).
Paste your Anthropic API key and click Save API Key.
Fill in Product Info — name, type, and description of your product.
Paste your User Needs JSON array.
Paste your Product Requirements JSON array.
Click Back. You are ready to run assessments.

Tip

All settings persist across sessions using Atlassian Forge Storage. You only need to configure them once. They survive app redeployments.

Running an assessment

The panel is opened per-ticket — you choose when to run it. It is not automatic on every issue.

Open a Jira issue you want to triage.
Click "View app actions" → "SaMD Complaint Risk Assessment".
The panel opens showing the issue ID, title, and description.
Click Run Risk Assessment.
The button changes to "Running risk assessment..." and disables. Steps 2 (Probability) and 3 (Severity) run in parallel. The full pipeline runtime depends on size of context window.
The result appears below the button.

What data is sent to the AI

Each time you click Run, the app fetches a fresh copy of the issue — title, description, reporter, and all comments. Previously generated risk assessment comments on the ticket are excluded from the context so they don't influence future runs.

Reading results

NON-DEFECT result

If the AI determines the issue does not represent a deviation from any User Need or Product Requirement, the result shows a grey NON-DEFECT badge. No probability, severity, or risk score is shown — no further triage action is needed for this ticket.

DEFECT result

If the issue is classified as a defect, the result shows:

Defect classification — which User Needs and/or Product Requirements are violated (by their configured IDs)
Probability score — a 1–5 score with the label from your configured scale and a rationale
Severity score — a 1–5 score with the label from your configured scale and a rationale
Risk level — LOW, MEDIUM, or HIGH, derived from your configured 5×5 risk matrix
Disposition — the recommended action for this risk level:
- LOW — Complaint, moderate timeline
- MEDIUM — Evaluate for CAPA escalation
- HIGH — CAPA Required
Evidence — citations showing which specific fields or comments influenced each conclusion

Auto-posted comment

If the Auto-post setting is enabled (the default), a formatted comment is posted to the Jira ticket after each run. The comment includes the risk level, disposition, probability, severity, evidence, and a timestamp. This creates an audit trail directly on the ticket.

Important

AI-generated results are a decision-support tool, not a final determination. A qualified engineer or quality manager should review and confirm the assessment before it is used in any regulated process.

Settings reference

Open Settings from the top-right button inside the panel. All fields save independently. A yellow dot (●) indicates unsaved changes in a field. A green checkmark (✓) indicates the field is saved.

If you click Back with unsaved changes, you will see a warning listing which fields are dirty. You can choose to Save all changes, Leave without saving, or Keep editing.

Anthropic API Key

Your Anthropic API key is required for the pipeline to run. The app cannot function without it.

Paste your key (format: sk-ant-...) into the password field and click Save API Key.
Once saved, the key is stored in Atlassian Forge Storage and is not displayed in plain text again. How is my API key stored?
The status shows either "✓ API key is saved" or "No API key saved yet."
Click Remove to delete the stored key.

Where to get a key

Create an API key at console.anthropic.com. API usage is billed to your Anthropic account — typically a few cents per assessment depending on ticket length.

Behaviour

A single checkbox controls whether the completed risk assessment is automatically posted as a Jira comment after each run.

Setting	Default	Effect when enabled
Auto-post risk assessment as a Jira comment	On	After each successful run, a formatted comment is posted to the ticket with the full result, scores, evidence, and timestamp.

This setting saves immediately when toggled — no Save button required.

Product Info

Three short text fields that tell the AI what product it is assessing. These are injected into every prompt and directly influence how the AI contextualises defect classification and risk scoring.

Field	Placeholder / example	What it does
Product Name	`MindBridge`	Used in the prompt header and in the auto-posted comment summary.
Product Type	`regulated mental health Software as Medical Device (SaMD)`	Tells the AI the regulatory classification and product category. Affects how it frames severity and harm.
Product Description	`a CBT-based therapeutic chatbot`	A brief description of what the product does. Helps the AI understand the therapeutic context when evaluating patient impact.

Click Save Product Info after editing.

Additional Context

A free-form text field. Whatever you write here is injected verbatim into every prompt — it supplements the structured fields above and is not parsed or validated.

Use this for:

Regulatory classification context (e.g. "This is a Class IIa device under EU MDR")
Risk tolerance guidance (e.g. "Our risk policy defaults to escalating borderline MEDIUM cases to HIGH when patient population is elderly")
Known product edge cases (e.g. "Issues related to the offline sync feature should be treated as higher severity due to data loss risk")
QMS-specific instructions (e.g. "Always flag issues involving the device's connection to external hardware as HIGH severity")

Click Save Additional Context after editing.

LLM tip

This is the most flexible field in the app. If you find the AI is consistently misjudging a class of issues, add an instruction here before fine-tuning your User Needs or Product Requirements.

User Needs

A JSON array defining the User Needs of your product. The defect classification step uses these to determine whether a complaint represents a failure of a capability the product is supposed to provide.

Required format:

[
  {
    "id": "UN-001",
    "description": "User shall be able to log in and access their account on all supported platforms."
  },
  {
    "id": "UN-002",
    "description": "User shall be able to save and retrieve their session progress across app sessions."
  }
]

Field	Required	Notes
`id`	Yes	A unique identifier. Appears in the assessment output under `failed_user_needs`. Use a consistent format like `UN-001`.
`description`	Yes	A clear statement of what the user should be able to do. Write these as "shall" statements.

Click Save after editing. The field validates that the input is valid JSON before saving — if you see "Invalid JSON — check your formatting", use a JSON validator to find the syntax error.

This field is required

If User Needs are not configured, the pipeline will return an error: "Product context not configured. Please add user needs and product requirements in Settings."

Product Requirements

A JSON array defining the Product Requirements (design outputs) of your product. The defect classification step checks whether the complaint indicates a failure to meet a specific requirement. The traces_to field links each requirement back to the User Need it satisfies.

Required format:

[
  {
    "id": "PR-001",
    "description": "The system shall support login on iOS 15 and above.",
    "traces_to": "UN-001"
  },
  {
    "id": "PR-002",
    "description": "The system shall persist session data and restore it on next launch without data loss.",
    "traces_to": "UN-002"
  }
]

Field	Required	Notes
`id`	Yes	Unique identifier. Appears in assessment output under `failed_requirements`. Use a consistent format like `PR-001`.
`description`	Yes	The specific, testable requirement. Write as "shall" statements.
`traces_to`	Recommended	The `id` of the User Need this requirement satisfies. Used to build the traceability chain for the AI. Should match a value from your User Needs list.

Click Save after editing.

Tip — granularity matters

The more specific and testable your requirements, the more precise the defect classification. Vague requirements like "the product shall be reliable" produce vague results. Requirement statements scoped to specific features or behaviours perform much better.

Probability Scale

Defines what each probability score (1–5) means in the context of your product. The AI uses these definitions when scoring how likely users are to encounter a defect during normal use.

Each level has two editable fields: a short Label and a longer Description. Level 1 is the lowest probability; Level 5 is the highest.

Default scale:

Level	Default label	Default description
1	Remote	unlikely to occur
2	Low	could occur but rare
3	Moderate	may occur occasionally
4	High	likely to occur
5	Very High	almost certain to occur

Edit the label and description for any level, then click Save Probability Scale.

Severity Scale

Defines what each severity score (1–5) means in the context of your product. The AI uses these definitions when assessing the potential harm to a patient or user if the defect occurs.

Each level has a Label and a Description. Level 1 is the least severe; Level 5 is the most severe.

Default scale:

Level	Default label	Default description
1	Negligible	no impact on therapeutic experience
2	Minor	slight inconvenience, user can continue
3	Moderate	disrupts session but user can recover
4	Major	prevents therapeutic function or causes distress
5	Critical	potential for clinical harm or safety event

Edit the label and description for any level, then click Save Severity Scale.

Tip — tailor these to your product

The default scales are written for a therapeutic SaMD. If your product is a diagnostic tool, a surgical planning aid, or a different device category, rewrite the severity descriptions to reflect harm pathways relevant to your specific intended use.

Risk Matrix

A 5×5 interactive grid that maps every combination of Severity (rows) and Probability (columns) to a risk level: LOW, MEDIUM, or HIGH.

How to read and edit the matrix:

Rows represent Severity levels — Severity 5 is at the top, Severity 1 at the bottom.
Columns represent Probability levels — Probability 1 is on the left, Probability 5 on the right.
Each cell shows the current risk level for that S×P combination.
Click any cell to cycle it through LOW → MEDIUM → HIGH → LOW.
The row and column labels reflect your configured Probability and Severity scale labels.
Click Save Risk Matrix to persist your changes.
Click Reset to ISO Default to restore the default matrix (see below).

Default (ISO 14971-aligned) risk matrix:

Severity ↓ / Probability →	P1 Remote	P2 Low	P3 Moderate	P4 High	P5 Very High
S5 Critical	HIGH	HIGH	HIGH	HIGH	HIGH
S4 Major	MEDIUM	HIGH	HIGH	HIGH	HIGH
S3 Moderate	LOW	MEDIUM	MEDIUM	HIGH	HIGH
S2 Minor	LOW	LOW	MEDIUM	MEDIUM	HIGH
S1 Negligible	LOW	LOW	LOW	LOW	LOW

Troubleshooting

"Product context not configured" error

The pipeline requires both User Needs and Product Requirements to be saved before it can run. Open Settings and paste valid JSON arrays into both fields, then click Save on each.

"No API key saved" error

Open Settings, paste your Anthropic API key into the API Key field, and click Save API Key. The key status indicator at the top of the field should show a green checkmark after saving.

"Invalid JSON — check your formatting" when saving User Needs or Product Requirements

The field requires valid JSON. Common issues:

Missing commas between objects in the array
Trailing comma after the last object (not valid in standard JSON)
Single quotes instead of double quotes
Missing closing bracket ]

Paste your JSON into jsonlint.com to identify the exact error.

Assessment takes longer than 30 seconds or times out

The pipeline makes three sequential (or partially parallel) API calls to Anthropic. Slowness is usually caused by Anthropic API latency under load. Try running again. If timeouts persist, check your API key usage limits in the Anthropic console.

Auto-posted comment does not appear on the ticket

Confirm that Auto-post risk assessment as a Jira comment is checked in Settings. Also confirm the app has write:jira-work permission — this is included in the standard installation. If you installed the app before this permission was added, run forge install --upgrade from your terminal.

Settings I saved are gone after redeployment

Settings are stored in Atlassian Forge Storage, not in the app bundle. They persist across deploys. If settings appear to be missing, check whether you are on the same Jira site (settings are scoped to the site the app is installed on).

FAQ

Does the app work on Jira Server or Data Center?

No. The app is built on Atlassian Forge, which only supports Jira Cloud.

Is my issue data stored anywhere?

No. Issue content (title, description, comments) is sent to Anthropic's API for processing and is not stored by this app after the API call completes. See the Privacy Policy for full details.

How much does it cost to run an assessment?

The app itself is free. You pay Anthropic for API usage. Assessment will be based on size of context window. Check anthropic.com/pricing for current rates.

Can I run multiple assessments on the same ticket?

Yes. Each click of Run Risk Assessment is independent — it fetches fresh issue data and runs a new pipeline. If auto-post is on, each run adds a new comment to the ticket. Previously posted comments are excluded from the AI context to avoid circular reasoning.

Can I use this for non-SaMD products?

Yes. The assessment framework is configurable. Replace the User Needs, Product Requirements, probability scale, severity scale, and risk matrix with values appropriate to your product type. The default severity scale is written for therapeutic SaMD — you should replace it if your product has different harm pathways.

The AI consistently over- or under-scores a type of issue. How do I fix it?

Try the following in order:

Add a specific instruction in Additional Context (e.g. "Login-related issues should always score at least Probability 3 due to daily user exposure").
Make the relevant Product Requirement more specific.
Adjust the severity or probability scale descriptions to better reflect your product's risk profile.
Adjust the risk matrix thresholds directly.