JSON / HTML report

[anerli]

Should have some ability to export JSON / HTML report of results, maybe with screenshots and history of what agent did in addition to test results

[anerli]

related to #11

[ashutosh-rath02]

Hey @anerli , here's what I am planning, please take a look.

Changes Proposed

• Add --output-json and --output-html flags to the CLI to export test results.
• Collect and include agent action history and screenshots for each test.
• Planning to use the existing screenshot capture in WebHarness, and aggregate action logs from the agent’s event system.
• For HTML, will use a simple template (probably string-based, not a framework) to show results, actions, and screenshots.

I had few questions for a more clean approach

• When should screenshots be taken for the report—after every action, only on failure, or both?
• Any preference for where screenshots are saved or how they’re named?
• Should HTML reports embed screenshots (base64) or just link to files?
• Anything specific you want included in the JSON/HTML output besides results, actions, and screenshots?

[anerli]

Yeah I'm not sure what exactly we want out of a json output yet, @ewired might be working on some changes for machine-readable output which may be related. really for that we want something that can be passed into CI runners etc.

maybe on this issue better to start with HTML output?
self-contained html might be nice but also could be large filesize if images are all embedded as b64 strs

i think the content should be actions, agent thoughts, ideally checks as well (may not have a hook for that), and like overall test state, in correct groups etc.
might be nice to show screenshots after each action.. but as self-contained html might get too heavy. there is also lightweight replay implementations like rrweb that are more purpose built for showing replays of what happened in a lightweight representation that could be worth looking into for this

[ashutosh-rath02]

This is something I got out for a json output @anerli , will try out the html part with replay screenshots