Lighthouse Custom Gatherer: Unable to Collect Button Accessibility Text Due to Runtime.evaluate Error

[kal-purush]

Description

I am trying to write a custom Lighthouse audit that checks the accessibility text of all buttons on a webpage. To achieve this, I created a custom Gatherer that collects all button elements. However, when running the audit, I encounter the following error:

🚨 Error Output in Lighthouse Console:

LH:LinksInButtons:error Runtime.evaluate exception LH:LinksInButtons:error Expression: (() => { return Array.from(document.querySelectorAll("button")).map(button => ({ text: button.textCo LH:LinksInButtons:error ---- (elided) LH:LinksInButtons:error Parse error at: 11:270 LH:LinksInButtons:error SyntaxError: missing ) after argument list +1ms

📌 My Current Gatherer Code:

import { Gatherer } from 'lighthouse';

class ButtonGatherer extends Gatherer {
  meta = {
    supportedModes: ['navigation', 'timespan'],
  };

  async getArtifact(context) {
    const { driver } = context;
    const { executionContext } = driver;

    const result = await executionContext.evaluateAsync(
      '(() => { return Array.from(document.querySelectorAll("button")).map(button => ({ text: button.textContent ? button.textContent.trim() : "", id: button.id || null, class: button.className || null, type: button.getAttribute("type") || null })); })();'
    );

    console.log("Live Query Results:", result); // Debugging log in Lighthouse console
    return result;
  }
}

✅ Simplified Code That Works (But Wrong Output)

import { Gatherer } from 'lighthouse';

class LinksGatherer extends Gatherer {
  meta = {
    supportedModes: ['navigation', 'timespan'],
  };

  async getArtifact(context) {
    // const { driver } = context;
    const {driver, page} = context;
    const {executionContext} = driver;
    const buttons = await executionContext.evaluateAsync(`document.querySelectorAll('button', [role="button"])`);
    return buttons;
  }
}

export default LinksGatherer;

After simplifying the code, I no longer get errors, but the output is different than what I expect.

Expected Output (From DevTools Console Query)

When I manually run:

document.querySelectorAll('button', [role="button"])

I get

NodeList(4) [ button.non-usa-accordion-button.usa-banner-button, button#account_info.header-button, button.ncbi-close-button, button#search.button_search.nowrap
 ]
​
0: <button class="non-usa-accordion-button usa-banner-button" aria-expanded="false" aria-controls="gov-banner-top" type="button">​
1: <button id="account_info" class="header-button" style="display:none" aria-controls="account_popup" type="button" aria-haspopup="true" aria-expanded="false">​
2: <button class="ncbi-close-button" data-ga-action="close_menu" data-ga-label="account_menu" type="button">​
3: <button id="search" class="button_search nowrap" type="submit" cmd="go">
​
length: 4

But from Lighthouse I get

Key: 0
  Click Events: [
  {
    type: 'click',
    origType: 'click',
    handler: { guid: 32 },
    guid: 32,
    namespace: ''
  }
]
Key: 1
  Click Events: [
  {
    type: 'click',
    origType: 'click',
    handler: { guid: 28 },
    guid: 28,
    namespace: ''
  }
]
Key: 2
  No jQuery events found
Key: 3
  Click Events: [
  {
    type: 'click',
    origType: 'click',
    data: null,
    handler: { guid: 25 },
    guid: 25,
    namespace: ''
  }
]
  Keypress Events: [
  {
    type: 'keypress',
    origType: 'keypress',
    data: null,
    handler: { guid: 25 },
    guid: 25,
    namespace: ''
  }
]

❓ Questions

• How can I fix the Runtime.evaluate error? I suspect it is related to Lighthouse’s sandboxing or how evaluateAsync handles multi-line queries.
• How can I get the correct expected output, similar to DevTools? Currently, I do not see all buttons or their correct text content.
• How can I collect all aria-* attributes from the page? Specifically, I need aria-label, aria-labelledby, and aria-describedby.

[connorjclark]

evaluateAsync is an older function for running JS inside the page from a gatherer. Can you try the newer function (evaluate) and report back? It should be something like this (untested):

const result = await executionContext.evaluate(function() {
      return Array.from(document.querySelectorAll("button"))
        .map(button => ({
          text: button.textContent ? button.textContent.trim() : "",
          id: button.id || null,
          class: button.className || null,
          type: button.getAttribute("type") || null,
        }));
    }, {args: []});
);

I think there's a bug in the deprecated function evaluateAsync that makes it not work for all JS expressions. Perhaps the newer one will work better.