Spec file formats

Feature support matrix

`local.type`	Push	Pull	AI summary	Outline expansion	Attachments	Path auto-tags
`gherkin`	✓	✓	—	✓	✓	✓
`reqnroll`	✓	✓	—	✓	✓	✓
`markdown`	✓	✓	—	—	✓	✓
`csv`	✓	—	—	—	—	—
`excel`	✓	—	—	—	—	—
`csharp`	✓	—	✓	—	—	✓
`java`	✓	—	✓	—	—	✓
`python`	✓	—	✓	—	—	✓
`javascript`	✓	—	✓	—	—	✓
`playwright`	✓	—	✓	—	—	✓
`puppeteer`	✓	—	✓	—	—	✓
`cypress`	✓	—	✓	—	—	✓
`testcafe`	✓	—	✓	—	—	✓
`detox`	✓	—	✓	—	—	✓
`espresso`	✓	—	✓	—	—	✓
`xcuitest`	✓	—	✓	—	—	✓
`flutter`	✓	—	✓	—	—	✓
`robot`	✓	—	—	—	—	✓

Column notes:

Pull — ado-sync pull can create/update local files from Azure Test Cases.
AI summary — sync.ai auto-generates title, description and steps from the test function body. Requires a non-heuristic provider for best results; heuristic regex matching works for all code-based types.
Outline expansion — Scenario Outline / Examples tables create a single parametrized TC (not one TC per row).
Attachments — sync.attachments file-attachment sync via @attachment: tags.
Path auto-tags — directories starting with @ (e.g. specs/@smoke/) are automatically applied as tags.

Gherkin `.feature`

Set local.type: "gherkin". Standard Gherkin syntax is supported:

Feature / Scenario / Scenario Outline / Background
Given / When / Then / And / But
Feature-level and scenario-level tags
Scenario Outline with Examples tables — creates a single parametrized Test Case in Azure, not one TC per row
Inline data tables (synced as sub-steps or plain text — see Format configuration)

Feature: Checkout

  Background:
    Given I am on the storefront

  @smoke
  @tc:1041
  Scenario: Add item and complete checkout
    Given I am logged in as "standard_user"
    When I add "Sauce Labs Backpack" to the cart
    And I proceed through checkout with name "Test User" and zip "12345"
    Then I see the order confirmation page

  @tc:1042
  Scenario Outline: Checkout with different users
    Given I am logged in as "<user>"
    When I complete a checkout
    Then the result is "<result>"

    Examples:
      | user                    | result  |
      | standard_user           | success |
      | performance_glitch_user | success |

Markdown `.md`

Set local.type: "markdown". Each ### heading is one test case. Scenarios are separated by ---.

# My Feature Test Plan

## Test scenarios

### Login with valid credentials
@tc:1042 @smoke

Assumption: Fresh browser session.

Steps:
1. Navigate to https://example.com/login
2. Enter username "admin" and password "secret"
3. Click the Login button

Expected results:
- The dashboard page is shown
- The username appears in the top navigation

---

### Login with invalid credentials

Steps:
1. Navigate to https://example.com/login
2. Enter username "wrong" and password "wrong"
3. Click the Login button

Expected results:
- An error message "Invalid credentials" is displayed
- The user remains on the login page

---

Sections recognised (case-insensitive): Steps:, Expected results:. All other prose is captured as the test case description. Heading number prefixes (### 1. Title or ### Title) are both supported.

Playwright test-plan markdown

Markdown files generated by the Playwright MCP agent are fully supported. Set local.type: "markdown" and point local.include at the generated files.

C# MSTest / NUnit `.cs`

Set local.type: "csharp". Both MSTest and NUnit frameworks are supported. Each test method becomes one Test Case.

Framework attribute mapping

Concern	MSTest	NUnit
Test marker	`[TestMethod]`	`[Test]`
Category / tag	`[TestCategory("name")]`	`[Category("name")]`
Custom property	`[TestProperty("key","val")]`	`[Property("key","val")]`
TC ID writeback	`[TestProperty("tc","ID")]`	`[Property("tc","ID")]`

The framework is auto-detected per method — mixed files (e.g. a shared helper class) are handled correctly.

Source mapping (both frameworks)

C# source	Azure TC field
XML doc `<summary>` first line	TC Title
Numbered lines `N. text` in summary	TC Steps (action)
Numbered lines `N. Check: text`	TC Steps (expected result, when `useExpectedResult: true`)
`[TestCategory("…")]` / `[Category("…")]`	TC Tags (string literals and `const string` constants resolved)
`[TestProperty("tc","ID")]` / `[Property("tc","ID")]`	TC ID (written back after first push)
`Namespace.Class.MethodName`	`AutomatedTestName` (for TRX result linking)

MSTest example

/// <summary>
/// Verify CoCounsel dialog opens on Edge
/// Test case: 1883058
/// 1. Sign in to Westlaw Edge
/// 2. Click CoCounsel link from page header
/// 3. Check: Verify CoCounsel dialog opens with correct title
/// 4. Check: Verify Close button is displayed
/// </summary>
[TestMethod]
[TestProperty("tc", "1883058")]   // written back by ado-sync after first push
[TestCategory("EdgeAalp")]
[TestCategory("EdgeAalpSmoke")]
public void EdgeAalpAccessTest()
{
    // ...
}

NUnit example

/// <summary>
/// Verify login with valid credentials
/// 1. Navigate to the login page
/// 2. Enter username and password
/// 3. Click the Login button
/// 4. Check: Dashboard page is shown
/// 5. Check: Username appears in the top navigation
/// </summary>
[Test]
[Property("tc", "2001")]   // written back by ado-sync after first push
[Category("Smoke")]
[Category("Authentication")]
public void LoginWithValidCredentials()
{
    // ...
}

Recommended config

{
  "local": {
    "type": "csharp",
    "include": ["**/RegressionTests/**/*.cs"],
    "exclude": ["**/*BaseTest.cs", "**/*BaseFixture.cs", "**/*Helper.cs"]
  },
  "sync": {
    "markAutomated": true,
    "format": { "useExpectedResult": true }
  }
}

With useExpectedResult: true, Check: lines go into the Expected Result column of each TC step. With markAutomated: true, the TC's Associated Automation is set to the fully-qualified method name (Namespace.Class.Method), enabling publish-test-results to link TRX run outcomes back to TCs.

Notes

Constants — ado-sync resolves const string declarations within the same file. Constants defined in a base class are not resolved; use string literals for reliable tagging.
pull is not supported for C# files. Only push and publish-test-results apply.
Base classes / fixtures — exclude them from local.include to avoid treating non-test methods as test cases.
NUnit [TestCase] — parameterised data rows ([TestCase(1, "foo")]) are not expanded into separate TCs. Only the [Test] marker is treated as the test definition.

Java JUnit / TestNG `.java`

Set local.type: "java". Supports JUnit 4, JUnit 5, and TestNG (including Selenium-based tests). Each @Test-annotated method becomes one Test Case.

Framework attribute mapping

Concern	JUnit 4	JUnit 5	TestNG
Test marker	`@Test` (`org.junit.Test`)	`@Test` (`org.junit.jupiter.api.Test`)	`@Test` (`org.testng.annotations.Test`)
Tag / category	`@Category(Smoke.class)`	`@Tag("smoke")`	`groups = {"smoke"}` in `@Test`
TC ID writeback	`// @tc:ID` above `@Test`	`@Tag("tc:ID")` above `@Test`	`// @tc:ID` above `@Test`

The framework is auto-detected per file from import statements — no config required.

Source mapping (all frameworks)

Java source	Azure TC field
Javadoc `/** ... */` first non-numbered line	TC Title
Numbered lines `N. text` in Javadoc	TC Steps (action)
Numbered lines `N. Check: text` in Javadoc	TC Steps (expected result, when `useExpectedResult: true`)
`@Category`, `@Tag`, `groups` in `@Test`	TC Tags
`// @tc:ID` or `@Tag("tc:ID")`	TC ID (written back after first push)
`com.example.MyClass.myMethod`	`AutomatedTestName` (for JUnit XML result linking)

JUnit 5 example

import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;

class CheckoutTests {

    /**
     * Add item and complete checkout
     * 1. Sign in as standard_user
     * 2. Add "Sauce Labs Backpack" to cart
     * 3. Proceed through checkout
     * 4. Check: Order confirmation page is shown
     */
    @Tag("tc:1041")        // written back by ado-sync after first push
    @Tag("smoke")
    @Test
    void addItemAndCompleteCheckout() {
        // ...
    }
}

JUnit 4 example

import org.junit.Test;
import org.junit.experimental.categories.Category;

public class LoginTests {

    /**
     * Login with valid credentials
     * 1. Navigate to the login page
     * 2. Enter username and password
     * 3. Check: Dashboard page is shown
     */
    // @tc:1042               // written back by ado-sync after first push
    @Test
    @Category(Smoke.class)
    public void loginWithValidCredentials() {
        // ...
    }
}

TestNG example

import org.testng.annotations.Test;

public class SearchTests {

    /**
     * Search returns relevant results
     * 1. Open the search page
     * 2. Enter "junit 5" in the search box
     * 3. Check: Results list contains at least one entry
     */
    // @tc:1043               // written back by ado-sync after first push
    @Test(groups = {"regression", "search"})
    public void searchReturnsRelevantResults() {
        // ...
    }
}

Recommended config

{
  "local": {
    "type": "java",
    "include": ["**/src/test/**/*.java"],
    "exclude": ["**/*BaseTest.java", "**/*Helper.java", "**/*Utils.java"]
  },
  "sync": {
    "markAutomated": true
  }
}

Notes

pull is not supported for Java files. Only push and publish-test-results apply.
Abstract base test classes — exclude them from local.include to avoid treating base methods as test cases.
JUnit 5 @Tag — the org.junit.jupiter.api.Tag import must be present or the file must already use Jupiter annotations; ado-sync adds the tag annotation but not the import if it's missing.
TestNG groups — groups = {"smoke", "regression"} inside @Test(...) are extracted as TC tags.
@ParameterizedTest / @DataProvider — parameterised data rows are not expanded into separate TCs; only the method definition is treated as the test case.

Python pytest `.py`

Set local.type: "python". Each def test_* function — at module level or inside a class — becomes one Test Case. No extra dependencies required beyond pytest itself.

Source mapping

Python source	Azure TC field
Docstring first non-numbered line	TC Title
Numbered lines `N. text` in docstring	TC Steps (action)
Numbered lines `N. Check: text` in docstring	TC Steps (expected result, when `useExpectedResult: true`)
`@pytest.mark.<tag>` decorators	TC Tags (built-in pytest marks excluded)
`@pytest.mark.tc(ID)`	TC ID (written back after first push)
`module.path.ClassName.test_method`	`AutomatedTestName` (for JUnit XML result linking)

Example

import pytest

class TestCheckout:

    @pytest.mark.tc(1041)      # written back by ado-sync after first push
    @pytest.mark.smoke
    def test_add_item_and_complete_checkout(self):
        """
        Add item and complete checkout
        1. Sign in as standard_user
        2. Add Sauce Labs Backpack to cart
        3. Proceed through checkout
        4. Check: Order confirmation page is shown
        """
        # ...

    @pytest.mark.tc(1042)
    @pytest.mark.regression
    def test_checkout_with_invalid_card(self):
        """
        Checkout fails with invalid card
        1. Add an item to the cart
        2. Enter an invalid credit card number
        3. Click Place Order
        4. Check: Error message is displayed
        """
        # ...

Recommended config

{
  "local": {
    "type": "python",
    "include": ["tests/**/*.py"],
    "exclude": ["tests/conftest.py", "tests/**/fixtures.py"]
  },
  "sync": {
    "markAutomated": true
  }
}

Notes

pull is not supported for Python files. Only push and publish-test-results apply.
Built-in pytest marks (parametrize, skip, skipif, xfail, usefixtures, filterwarnings) are not pushed as TC tags.
@pytest.mark.parametrize — parameterised data rows are not expanded into separate TCs; only the function definition is treated as the test case.
Class hierarchy — only the immediately enclosing class is used for the automatedTestName. Nested classes are supported.

JavaScript / TypeScript (Jest, Jasmine, WebdriverIO) `.js` / `.ts`

Set local.type: "javascript". Supports Jest, Jasmine, and WebdriverIO (which uses Jest or Jasmine as its runner). All three share the same describe()/it()/test() API, so a single parser handles all of them.

Playwright Test (@playwright/test) has its own type: use local.type: "playwright" instead — see below. Cucumber .feature files are handled by local.type: "gherkin" — not this type.

Detected test functions

Function	Description
`it(title, fn)`	Standard test
`test(title, fn)`	Alias for `it`
`it.only` / `test.only`	Focused test
`it.skip` / `test.skip`	Skipped test (still synced)
`xit` / `xtest`	Jasmine-style skip
`it.concurrent` / `test.concurrent`	Concurrent test

Source mapping

JavaScript/TS source	Azure TC field
JSDoc `/** ... */` first non-numbered line	TC Title
Numbered lines `N. text` in JSDoc	TC Steps (action)
Numbered lines `N. Check: text` in JSDoc	TC Steps (expected result, when `useExpectedResult: true`)
`// @tags: smoke, regression` above `it()`	TC Tags (comma-separated list)
`// @smoke` above `it()` (single-word)	TC Tag
`// @tc:ID` above `it()`	TC ID (written back after first push)
`{basename} > {describe} > {it title}`	`AutomatedTestName` (Jest report format)

Example

describe('Checkout', () => {

  /**
   * Add item and complete checkout
   * 1. Sign in as standard_user
   * 2. Add Sauce Labs Backpack to cart
   * 3. Proceed through checkout
   * 4. Check: Order confirmation page is shown
   */
  // @tc:1041               // written back by ado-sync after first push
  // @tags: smoke, regression
  it('adds item and completes checkout', async () => {
    // ...
  });

  // @tc:1042
  // @smoke
  it('shows error on invalid card', async () => {
    // ...
  });

});

Recommended config

{
  "local": {
    "type": "javascript",
    "include": ["src/**/*.spec.ts", "tests/**/*.test.js"],
    "exclude": ["**/*.helper.ts", "**/*.fixture.ts"]
  },
  "sync": {
    "markAutomated": true
  }
}

Notes

pull is not supported for JavaScript/TypeScript files. Only push applies.
Dynamic titles — it / test calls with template literals or computed values are skipped. Use string literals for reliable syncing.
describe nesting — arbitrarily deep describe() nesting is supported. All enclosing describe titles are included in the automatedTestName.
Path-based auto-tagging — directory segments starting with @ (e.g. tests/@smoke/) are automatically applied as tags on all tests inside.

Playwright Test `.js` / `.ts`

Set local.type: "playwright". Supports Playwright Test (@playwright/test) for both JavaScript and TypeScript. Each test() call becomes one Test Case.

Jest/Jasmine/WebdriverIO tests use local.type: "javascript" — not this type.

Detected test functions

Function	Description
`test(title, fn)`	Standard test
`test.only`	Focused test
`test.skip`	Skipped test (still synced)
`test.fixme`	Test marked as broken (still synced, tagged `wip`)
`test.fail`	Test expected to fail (still synced)
`test.describe`	Nested describe block
`test.describe.parallel`	Parallel describe block
`test.describe.serial`	Serial describe block

ID tagging — native annotation (recommended)

Playwright Test has a built-in annotation API. Use it to attach the TC ID directly in the test definition — no comments needed:

// Single annotation (most common)
test('completes checkout', {
  annotation: { type: 'tc', description: '1041' },
  tag: '@smoke',
}, async ({ page }) => { ... });

// Tag-style ID (alternative)
test('cart badge shows count', {
  tag: ['@tc:1043', '@smoke'],
}, async ({ page }) => { ... });

// Array form — combine TC ID with other annotations
test.fixme('promo code applies discount', {
  annotation: [
    { type: 'tc', description: '1042' },
    { type: 'issue', description: 'promo-code not yet implemented' },
  ],
  tag: '@wip',
}, async ({ page }) => { ... });

Writeback — on the first push, ado-sync injects { annotation: { type: 'tc', description: 'N' } } into the test options object. Subsequent pushes update the description in place.

Comment fallback — // @tc:ID above test() is still recognised and written for edge cases where the options object cannot be parsed (e.g., multi-line title spanning several lines).

Source mapping

Playwright source	Azure TC field
JSDoc `/** ... */` first non-numbered line	TC Title
Numbered lines `N. text` in JSDoc	TC Steps (action)
Numbered lines `N. Check: text` in JSDoc	TC Steps (expected result, when `useExpectedResult: true`)
`annotation: { type: 'tc', description: 'N' }`	TC ID (preferred)
`annotation: [{ type: 'tc', description: 'N' }, ...]`	TC ID (array form)
`tag: '@tc:N'` or `tag: ['@tc:N', ...]`	TC ID (tag form)
`tag: '@smoke'` etc.	TC Tags
`// @tags: smoke, regression` above `test()`	TC Tags (comment form)
`// @tc:ID` above `test()`	TC ID (comment fallback)
`{basename} > {describe} > {test title}`	`AutomatedTestName`

Priority: native annotation > native tag > // @tc:N comment.

TypeScript example

import { test, expect } from '@playwright/test';

test.describe('SauceDemo Checkout', () => {

  /**
   * Complete checkout with valid details
   * 1. Log in as standard_user
   * 2. Add Sauce Labs Backpack to cart
   * 3. Proceed through checkout and enter name and zip
   * 4. Check: Order confirmation page is displayed
   */
  test('completes checkout with valid details', {
    annotation: { type: 'tc', description: '1041' },
    tag: '@smoke',
  }, async ({ page }) => {
    // ...
  });

  test.fixme('promo code applies discount', {
    annotation: [
      { type: 'tc', description: '1042' },
      { type: 'issue', description: 'promo-code not yet implemented' },
    ],
    tag: '@wip',
  }, async ({ page }) => {
    // known issue — still synced to Azure
  });
});

test.describe.parallel('Cart assertions', () => {

  test('cart badge shows correct count', {
    annotation: { type: 'tc', description: '1043' },
  }, async ({ page }) => {
    // ...
  });
});

JavaScript example (comment fallback style)

const { test, expect } = require('@playwright/test');

test.describe('SauceDemo Login', () => {

  /**
   * Valid credentials redirect to inventory page
   * 1. Navigate to https://www.saucedemo.com
   * 2. Enter username "standard_user" and password "secret_sauce"
   * 3. Click the login button
   * 4. Check: URL contains "inventory.html"
   */
  // @tc:1044   ← comment style also works; ado-sync upgrades to annotation on next push
  test('valid credentials redirect to inventory', async ({ page }) => {
    // ...
  });
});

Recommended config

{
  "local": {
    "type": "playwright",
    "include": ["tests/**/*.spec.ts", "tests/**/*.spec.js"],
    "exclude": ["**/*.helper.ts", "**/*.fixture.ts"]
  },
  "sync": {
    "markAutomated": true
  }
}

Notes

pull is not supported for Playwright files. Only push applies.
test.describe nesting — arbitrarily deep nesting is supported. All enclosing describe titles are included in the automatedTestName.
test.fixme / test.fail — both are parsed and synced as normal test cases. Use tag: '@wip' on test.fixme to mark them in Azure.
Publishing results — set testResultFormat: playwrightJson when using publish-test-results.
Native annotation priority — annotation: { type: 'tc', … } is read before tag: which is read before // @tc:N comments. The tool writes back using native annotation on every sync.

Puppeteer `.js` / `.ts`

Set local.type: "puppeteer". Supports Puppeteer tests written with Jest or Mocha as the test runner. Puppeteer tests use the same describe() / it() / test() API as Jest, so this type is an alias for javascript — choose it for clarity when your project uses Puppeteer.

import puppeteer from 'puppeteer';

describe('SauceDemo Login', () => {

  /**
   * Valid credentials redirect to inventory page
   * 1. Navigate to https://www.saucedemo.com
   * 2. Enter "standard_user" into the username field
   * 3. Enter "secret_sauce" into the password field
   * 4. Click the login button
   * 5. Check: URL contains "inventory"
   */
  // @tc:1060
  it('redirects to inventory on valid login', async () => {
    // ...
  });
});

Recommended config

{
  "local": {
    "type": "puppeteer",
    "include": ["tests/**/*.test.ts"],
    "exclude": ["**/*.helper.ts"]
  },
  "sync": { "markAutomated": true }
}

Notes

ID writeback format: // @tc:12345 immediately above the it()/test() line.
pull is not supported for Puppeteer files — only push applies.
describe() nesting is fully supported.

Cypress `.cy.js` / `.cy.ts`

Set local.type: "cypress". Supports Cypress tests. Cypress uses describe() / context() / it() / specify() — all four are detected.

describe('SauceDemo Login', () => {

  context('valid credentials', () => {

    /**
     * Successful login redirects to inventory page
     * 1. Visit https://www.saucedemo.com
     * 2. Enter "standard_user" into the username field
     * 3. Enter "secret_sauce" into the password field
     * 4. Click the login button
     * 5. Check: URL contains "inventory"
     */
    // @tc:1070
    // @tags: smoke
    it('redirects to inventory', () => {
      // ...
    });

    // @tc:1071
    specify('locked out user sees error', () => {
      // ...
    });
  });
});

Recommended config

{
  "local": {
    "type": "cypress",
    "include": ["cypress/e2e/**/*.cy.ts", "cypress/e2e/**/*.cy.js"],
    "exclude": ["cypress/support/**"]
  },
  "sync": { "markAutomated": true }
}

Notes

context() is treated as a describe() alias (Cypress convention).
specify() is treated as an it() alias (Cypress convention).
ID writeback format: // @tc:12345 immediately above the it()/specify() line.
pull is not supported for Cypress files — only push applies.

TestCafe `.js` / `.ts`

Set local.type: "testcafe". Supports TestCafe tests using the fixture / test API.

ID tagging — native `test.meta()` (recommended)

TestCafe has a built-in .meta() method that attaches key-value metadata to a test. Use it to store the TC ID natively — no comments needed:

// Key-value form (recommended — clean and simple)
test.meta('tc', '1080')('redirects to inventory on valid login', async t => { ... });

// Object form (use when adding multiple metadata fields)
test.meta({ tc: '1081', priority: 'high' })('locked out user sees error', async t => { ... });

// Combined with test.skip
test.skip.meta('tc', '1082')('skipped test', async t => { ... });

Writeback — on the first push, ado-sync injects .meta('tc', 'N') between the test function and its title call. For example:

test('title', fn)  →  test.meta('tc', '12345')('title', fn)

Subsequent pushes update the existing .meta() value in place.

Comment fallback — // @tc:N above test() is still recognised.

Source mapping

TestCafe source	Azure TC field
JSDoc `/** ... */` first non-numbered line	TC Title
Numbered lines `N. text` in JSDoc	TC Steps (action)
Numbered lines `N. Check: text` in JSDoc	TC Steps (expected result)
`test.meta('tc', 'N')`	TC ID (preferred)
`test.meta({ tc: 'N' })`	TC ID (object form)
`// @tags: smoke, regression` above `test()`	TC Tags
`// @tc:N` above `test()`	TC ID (comment fallback)
`{basename} > {fixture} > {test title}`	`AutomatedTestName`

Example

fixture('SauceDemo Login')
  .page('https://www.saucedemo.com');

/**
 * Valid credentials redirect to inventory page
 * 1. Type "standard_user" into the username field
 * 2. Type "secret_sauce" into the password field
 * 3. Click the login button
 * 4. Check: URL contains "inventory"
 */
// @smoke
test.meta('tc', '1080')('redirects to inventory on valid login', async t => {
  // ...
});

test.meta({ tc: '1081', priority: 'high' })('locked out user sees error', async t => {
  // ...
});

// Comment style still works (written by ado-sync before meta support was added)
// @tc:1082
test.skip('skipped test — still synced', async t => { ... });

Recommended config

{
  "local": {
    "type": "testcafe",
    "include": ["tests/**/*.js", "tests/**/*.ts"],
    "exclude": ["tests/helpers/**"]
  },
  "sync": { "markAutomated": true }
}

Notes

The fixture() title is used as the group / suite name and included in automatedTestName.
test.skip() and test.only() are both parsed and synced.
pull is not supported for TestCafe files — only push applies.
JSDoc /** ... */ above a test.meta(...) call is parsed for TC title and numbered steps.

Appium

Appium tests are written in your language of choice. Use the local.type that matches the language:

Appium language binding	Use `local.type`	Notes
JavaScript / TypeScript (WebdriverIO, Mocha, Jest)	`javascript`	Detects `describe()`/`it()`/`test()`
Java (JUnit 4, JUnit 5, TestNG)	`java`	Detects `@Test` annotations
Python (pytest)	`python`	Detects `def test_*()` functions
C# (NUnit, MSTest)	`csharp`	Detects `[TestMethod]`/`[Test]` attributes

JavaScript/TypeScript Appium example:

// @tc:12345
// @smoke
describe('Login', () => {
  /**
   * User can log in with valid credentials
   *
   * 1. Navigate to the login screen
   * 2. Enter email into the email field
   * 3. Enter password into the password field
   * 4. Tap the login button
   * 5. Check: Home screen is displayed
   */
  it('can log in with valid credentials', async () => {
    await driver.execute('mobile: launchApp', { bundleId: 'com.example.app' });
    await $('~emailField').setValue('user@example.com');
    await $('~passwordField').setValue('secret');
    await $('~loginButton').click();
    await expect($('~homeScreen')).toBeDisplayed();
  });
});

Recommended ado-sync.json:

{
  "local": {
    "type": "javascript",
    "include": ["test/**/*.spec.ts"]
  },
  "sync": { "markAutomated": true }
}

Detox `.js` / `.ts`

Set local.type: "detox" for React Native end-to-end tests using Detox.

Detox uses the same Jest describe()/it() API as Jest — the existing JavaScript parser handles it.

// @tc:12345
// @smoke
describe('Login flow', () => {
  /**
   * User can log in with valid credentials
   *
   * 1. Launch the app
   * 2. Type email into the email field
   * 3. Tap the login button
   * 4. Check: Welcome screen is visible
   */
  it('logs in with valid credentials', async () => {
    await device.launchApp();
    await element(by.id('email')).typeText('user@example.com');
    await element(by.id('password')).typeText('secret');
    await element(by.id('login-btn')).tap();
    await expect(element(by.id('welcome-screen'))).toBeVisible();
  });
});

Recommended ado-sync.json:

{
  "local": {
    "type": "detox",
    "include": ["e2e/**/*.test.ts"],
    "exclude": ["e2e/setup/**"]
  },
  "sync": { "markAutomated": true }
}

Notes:

describe() blocks define the test group (same as Jest).
// @tc:12345 is inserted immediately above the it()/test() line on writeback.
pull is not supported for Detox files — only push.

Espresso `.java` / `.kt`

Set local.type: "espresso" for Android UI tests using Espresso.

Espresso tests use JUnit 4 @Test annotations — the same parser as java handles them.

@RunWith(AndroidJUnit4.class)
public class LoginInstrumentedTest {

    // @tc:12345
    // @smoke
    /**
     * User can log in with valid credentials
     *
     * 1. Type email into the email field
     * 2. Type password into the password field
     * 3. Click the login button
     * 4. Check: Welcome screen is displayed
     */
    @Test
    public void userCanLoginWithValidCredentials() {
        onView(withId(R.id.email)).perform(typeText("user@example.com"), closeSoftKeyboard());
        onView(withId(R.id.password)).perform(typeText("secret"), closeSoftKeyboard());
        onView(withId(R.id.login_button)).perform(click());
        onView(withId(R.id.welcome_screen)).check(matches(isDisplayed()));
    }
}

Kotlin (same @Test detection):

@RunWith(AndroidJUnit4::class)
class LoginInstrumentedTest {

    // @tc:12345
    @Test
    fun `user can login with valid credentials`() {
        onView(withId(R.id.email)).perform(typeText("user@example.com"), closeSoftKeyboard())
        onView(withId(R.id.login_button)).perform(click())
        onView(withId(R.id.welcome_screen)).check(matches(isDisplayed()))
    }
}

Recommended ado-sync.json:

{
  "local": {
    "type": "espresso",
    "include": ["app/src/androidTest/**/*.java", "app/src/androidTest/**/*.kt"],
    "exclude": ["**/*BaseTest.java", "**/*Helper.java"]
  },
  "sync": { "markAutomated": true }
}

Notes:

ID writeback format: // @tc:12345 immediately above the @Test annotation line.
Kotlin backtick method names (fun `my test name`()) are supported — the method name is used as the fallback title.
pull is not supported — only push.

XCUITest `.swift`

Set local.type: "xcuitest" for iOS and macOS UI automation tests using Apple's XCTest framework.

Tests are func test*() methods inside classes that extend XCTestCase.

import XCTest

class LoginTests: XCTestCase {

    // @tc:12345
    // @smoke
    /// User can log in with valid credentials
    ///
    /// 1. Launch the app
    /// 2. Enter email into the email field
    /// 3. Enter password into the password field
    /// 4. Tap the login button
    /// 5. Check: Welcome screen is visible
    func testUserCanLoginWithValidCredentials() {
        let app = XCUIApplication()
        app.launch()
        app.textFields["Email"].typeText("user@example.com")
        app.secureTextFields["Password"].typeText("secret")
        app.buttons["Log In"].tap()
        XCTAssertTrue(app.staticTexts["Welcome"].exists)
    }

    func testLoginFailsWithInvalidPassword() {
        let app = XCUIApplication()
        app.launch()
        app.textFields["Email"].typeText("user@example.com")
        app.secureTextFields["Password"].typeText("wrong")
        app.buttons["Log In"].tap()
        XCTAssertTrue(app.staticTexts["Invalid credentials"].exists)
    }
}

Recommended ado-sync.json:

{
  "local": {
    "type": "xcuitest",
    "include": ["UITests/**/*.swift"],
    "exclude": ["UITests/**/*Helper.swift", "UITests/**/*Base.swift"]
  },
  "sync": { "markAutomated": true }
}

Notes:

The enclosing class name is used as the test group (automatedTestName = FileName > ClassName > methodName).
Both /// triple-slash (Swift idiomatic) and /** ... */ block doc comments are parsed for TC title and numbered steps.
Method name → title fallback: testUserCanLogin → "User can login", test_submit_form → "Submit form".
ID writeback format: // @tc:12345 immediately above the func test*() line.
pull is not supported — only push.

Flutter `_test.dart`

Set local.type: "flutter" for Flutter widget tests and integration tests using the flutter_test package.

import 'package:flutter_test/flutter_test.dart';

void main() {
  group('Login', () {
    // @tc:12345
    // @smoke
    /// User can log in with valid credentials
    ///
    /// 1. Tap the email field
    /// 2. Enter the email address
    /// 3. Tap the password field
    /// 4. Enter the password
    /// 5. Tap the Login button
    /// 6. Check: Welcome screen is visible
    testWidgets('can log in with valid credentials', (WidgetTester tester) async {
      await tester.pumpWidget(MyApp());
      await tester.enterText(find.byKey(Key('email')), 'user@example.com');
      await tester.enterText(find.byKey(Key('password')), 'secret');
      await tester.tap(find.byKey(Key('login-btn')));
      await tester.pumpAndSettle();
      expect(find.text('Welcome'), findsOneWidget);
    });

    // @tc:12346
    testWidgets('shows error for invalid credentials', (WidgetTester tester) async {
      await tester.pumpWidget(MyApp());
      await tester.enterText(find.byKey(Key('email')), 'bad@example.com');
      await tester.enterText(find.byKey(Key('password')), 'wrong');
      await tester.tap(find.byKey(Key('login-btn')));
      await tester.pumpAndSettle();
      expect(find.text('Invalid credentials'), findsOneWidget);
    });
  });
}

Recommended ado-sync.json:

{
  "local": {
    "type": "flutter",
    "include": ["test/**/*_test.dart", "integration_test/**/*_test.dart"]
  },
  "sync": { "markAutomated": true }
}

Notes:

group('title', () { ... }) is the describe equivalent. Nested groups are supported.
testWidgets(), test(), and testUI() (integration_test alias) are all detected.
The file base name is stripped of _test.dart / .dart suffixes for automatedTestName.
Both /// triple-slash and /** ... */ block doc comments are parsed.
ID writeback format: // @tc:12345 immediately above the testWidgets()/test() line.
pull is not supported — only push.

Robot Framework `.robot`

Set local.type: "robot". Supports Robot Framework test suites. Each test case in the *** Test Cases *** or *** Tasks *** section becomes one Azure Test Case.

Source mapping

Robot Framework source	Azure TC field
Test case name (non-indented line)	TC Title
`[Documentation]` row	TC Description
Numbered lines `N. text` in `[Documentation]`	TC Steps (action)
Numbered lines `N. Check: text` in `[Documentation]`	TC Steps (expected result, when `useExpectedResult: true`)
`[Tags]` values (excluding `tc:N`)	TC Tags
`[Tags] tc:N`	TC ID (preferred; written back after first push)
`# @tc:N` comment immediately above the test name	TC ID (comment fallback)
Test case name	`AutomatedTestName` (for result linking)

Example

*** Settings ***
Library    SeleniumLibrary

*** Test Cases ***
# @tc:1041
Add Item And Complete Checkout
    [Documentation]    Add item and complete checkout
    ...    1. Sign in as standard_user
    ...    2. Add Sauce Labs Backpack to cart
    ...    3. Proceed through checkout
    ...    4. Check: Order confirmation page is shown
    [Tags]    tc:1041    smoke
    Open Browser    ${URL}    chrome
    Input Text      id:user-name    standard_user
    Input Password  id:password     secret_sauce
    Click Button    id:login-button

User Login Fails With Invalid Credentials
    [Documentation]    Login fails with invalid credentials
    ...    1. Navigate to the login page
    ...    2. Enter an invalid username and password
    ...    3. Click Login
    ...    4. Check: Error message is displayed
    [Tags]    regression
    Open Browser    ${URL}    chrome
    Input Text      id:user-name    bad_user
    Input Password  id:password     wrong
    Click Button    id:login-button

On the first push, ado-sync inserts tc:N into the [Tags] row. If no [Tags] row exists, one is created immediately after the test name line.

ID writeback

Location	Format
`[Tags]` row (preferred)	`tc:12345` value added/updated in the existing `[Tags]` row
No `[Tags]` row	`[Tags] tc:12345` inserted after the test name line

The # @tc:N comment form above the test name is also recognised on read but is not used for writeback — [Tags] is always the authoritative location after a push.

Path-based auto-tagging

Directory segments starting with @ are automatically applied as tags on all test cases inside:

tests/
  @smoke/
    login.robot       ← all test cases get tag 'smoke'
  @regression/
    checkout.robot    ← all test cases get tag 'regression'

Recommended config

{
  "local": {
    "type": "robot",
    "include": ["tests/**/*.robot"],
    "exclude": ["tests/resources/**", "tests/keywords/**"]
  },
  "sync": {
    "markAutomated": true
  }
}

Notes

pull is not supported for Robot Framework files. Only push and publish-test-results apply.
*** Tasks *** sections are treated the same as *** Test Cases ***.
Keyword lines — all indented non-settings rows are treated as TC steps if no numbered steps are found in [Documentation].
[Setup] / [Teardown] — these settings rows are skipped; only keyword steps and [Documentation] / [Tags] are used.

CSV `.csv`

Set local.type: "csv" to parse Azure DevOps / SpecSync tabular CSV exports.

Expected column layout (9 columns):

Col	Field	Description
A (0)	ID	Azure Test Case ID — empty for new, filled after first push
B (1)	Work Item Type	Always `Test Case` (ignored)
C (2)	Title	`Scenario: My test` — non-empty on header row, empty on step rows
D (3)	Test Step	Step number — empty on header row
E (4)	Step Action	Step text, optionally prefixed with a Gherkin keyword
F (5)	Step Expected	Expected result text (optional)
G–I (6–8)	Area Path, Assigned To, State	Preserved but not used

The first row (column headers) is always skipped automatically.

Note: ado-sync pull is not supported for CSV files. Only push is supported.

Excel `.xlsx`

Set local.type: "excel". Uses the same 9-column layout as CSV. ado-sync reads the raw worksheet XML (no external xlsx library) and handles both t="str" and t="inlineStr" cells from Azure DevOps exports.

Note: ado-sync pull is not supported for Excel files. Only push is supported.

ID tags

After a first push, ado-sync writes the Azure TC ID back into the local file.

Format	Location
Gherkin	`@tc:12345` tag on its own line above the `Scenario:` line
Markdown	`@tc:12345` tag on the line immediately after the `### heading`
C# MSTest	`[TestProperty("tc", "12345")]` attribute on the test method
C# NUnit	`[Property("tc", "12345")]` attribute on the test method
Java JUnit 5	`@Tag("tc:12345")` annotation above `@Test`
Java JUnit 4 / TestNG	`// @tc:12345` comment on the line above `@Test`
Python pytest	`@pytest.mark.tc(12345)` decorator above `def test_*`
JavaScript/TS	`// @tc:12345` comment on the line above `it()`/`test()`
Playwright	`// @tc:12345` comment on the line above `test()`
Robot Framework	`tc:12345` value in the `[Tags]` row of the test case body
CSV	Numeric ID in column A of the matching title row
Excel	Numeric ID in cell A of the matching title row

Custom prefix

{ "sync": { "tagPrefix": "azure" } }

Result: @azure:12345 in both Gherkin and Markdown files.

Warning: Changing the prefix on an existing project means existing ID tags are no longer recognised. Do a project-wide find-and-replace before changing.

Tag filtering

All commands accept --tags to limit which scenarios are processed. Uses the standard Cucumber tag expression language.

ado-sync push --tags "@smoke"
ado-sync push --tags "@smoke and not @wip"
ado-sync pull --tags "@regression or @critical"

Tags are evaluated against all tags on a scenario, including:

Tags on the Feature block (Gherkin)
Tags on the Scenario / Scenario Outline block
Tags on individual Examples tables
Tags on the Markdown tag line (same ### heading line or line below it)

Path-based auto-tags — directory segments starting with @ are automatically applied as tags:

specs/
  @smoke/
    login.feature       ← all scenarios get tag 'smoke'
  @regression/
    @slow/
      checkout.feature  ← all scenarios get tags 'regression' and 'slow'

`local.condition` — permanent filter

Use local.condition in config to permanently exclude scenarios without needing --tags on every command:

{ "local": { "condition": "@done and not (@ignored or @planned)" } }

This filter is applied before --tags, so it acts as a baseline inclusion gate.

Work item linking

Tags matching a configured links prefix are synced as Azure DevOps work item relations on the Test Case.

Config

{
  "sync": {
    "links": [
      { "prefix": "story",  "relationship": "System.LinkTypes.Related" },
      { "prefix": "bug",    "relationship": "System.LinkTypes.Related" },
      { "prefix": "req",    "relationship": "Microsoft.VSTS.Common.TestedBy-Reverse" }
    ]
  }
}

Usage in Gherkin

  @tc:1042 @story:555 @bug:789
  Scenario: User can add items to cart
    ...

Usage in Markdown

### User can add items to cart
@tc:1042 @story:555 @bug:789

Steps:
1. Add an item to the cart
...

On each push, relations are synced: new links are added, stale links (whose tag was removed) are deleted. The relationship value is the ADO relation type — "System.LinkTypes.Related" is the default if omitted.

Attachments

Files can be attached to Azure Test Cases via tags. See Attachments.

Feature support matrix​

Gherkin .feature​

Markdown .md​

Playwright test-plan markdown​

C# MSTest / NUnit .cs​

Framework attribute mapping​

Source mapping (both frameworks)​

MSTest example​

NUnit example​

Recommended config​

Notes​

Java JUnit / TestNG .java​

Framework attribute mapping​

Source mapping (all frameworks)​

JUnit 5 example​

JUnit 4 example​

TestNG example​

Recommended config​

Notes​

Python pytest .py​

Source mapping​

Example​

Recommended config​

Notes​

JavaScript / TypeScript (Jest, Jasmine, WebdriverIO) .js / .ts​

Detected test functions​

Source mapping​

Example​

Recommended config​

Notes​

Playwright Test .js / .ts​

Detected test functions​

ID tagging — native annotation (recommended)​

Source mapping​

TypeScript example​

JavaScript example (comment fallback style)​

Recommended config​

Notes​

Puppeteer .js / .ts​

Recommended config​

Notes​

Cypress .cy.js / .cy.ts​

Recommended config​

Notes​

TestCafe .js / .ts​

ID tagging — native test.meta() (recommended)​

Source mapping​

Example​

Recommended config​

Notes​

Appium​

Detox .js / .ts​

Espresso .java / .kt​

XCUITest .swift​

Flutter _test.dart​

Robot Framework .robot​

Source mapping​

Example​

ID writeback​

Path-based auto-tagging​

Recommended config​

Notes​

CSV .csv​

Excel .xlsx​

ID tags​

Custom prefix​

Tag filtering​

local.condition — permanent filter​

Work item linking​

Config​

Usage in Gherkin​

Usage in Markdown​

Attachments​

Feature support matrix

Gherkin `.feature`

Markdown `.md`

Playwright test-plan markdown

C# MSTest / NUnit `.cs`

Framework attribute mapping

Source mapping (both frameworks)

MSTest example

NUnit example

Recommended config

Notes

Java JUnit / TestNG `.java`

Framework attribute mapping

Source mapping (all frameworks)

JUnit 5 example

JUnit 4 example

TestNG example

Recommended config

Notes

Python pytest `.py`

Source mapping

Example

Recommended config

Notes

JavaScript / TypeScript (Jest, Jasmine, WebdriverIO) `.js` / `.ts`

Detected test functions

Source mapping

Example

Recommended config

Notes

Playwright Test `.js` / `.ts`

Detected test functions

ID tagging — native annotation (recommended)

Source mapping

TypeScript example

JavaScript example (comment fallback style)

Recommended config

Notes

Puppeteer `.js` / `.ts`

Recommended config

Notes

Cypress `.cy.js` / `.cy.ts`

Recommended config

Notes

TestCafe `.js` / `.ts`

ID tagging — native `test.meta()` (recommended)

Source mapping

Example

Recommended config

Notes

Appium

Detox `.js` / `.ts`

Espresso `.java` / `.kt`

XCUITest `.swift`

Flutter `_test.dart`

Robot Framework `.robot`

Source mapping

Example

ID writeback

Path-based auto-tagging

Recommended config

Notes

CSV `.csv`

Excel `.xlsx`

ID tags

Custom prefix

Tag filtering

`local.condition` — permanent filter

Work item linking

Config

Usage in Gherkin

Usage in Markdown

Attachments