#7 Move away from HTML report uploading to XML report handling

Open
opened 1 year ago by kjhoerr · 2 comments
kjhoerr commented 1 year ago
Owner

This will require parsing more metadata from reports and generating our own HTML files. This feature design needs to be developed more.

Cons

  • This will make the coverage report server much less 'simple'.
  • More metadata may be required to be stored.

Pros

  • This will allow for a much greater number of coverage reporters to be used without need for supporting specific formats or uploading archives.
  • Currently the format handling is potentially dangerous for end-users as the current solution has a lot of surface area for XSS and other security issues or abuse.
  • Metadata about specific statistics will be easier to grab since the XML report will already be parsed to generate the HTML report file
  • The displayed reports will be more homogeneous across the server.

This would be a good and possibly necessary feature to implement when multiple users must be trusted to upload reports.

Edit: Workflow checklist:

This will require parsing more metadata from reports and generating our own HTML files. This feature design needs to be developed more. ### Cons * This will make the coverage report server much less 'simple'. * More metadata may be required to be stored. ### Pros * This will allow for a much greater number of coverage reporters to be used without need for supporting specific formats or uploading archives. * Currently the format handling is potentially dangerous for end-users as the current solution has a lot of surface area for XSS and other security issues or abuse. * Metadata about specific statistics will be easier to grab since the XML report will already be parsed to generate the HTML report file * The displayed reports will be more homogeneous across the server. This would be a good and possibly necessary feature to implement when multiple users must be trusted to upload reports. Edit: Workflow checklist: * [ ]  Replace POST API endpoint for HTML with XML. Functionally very similar, since the XML file will also be 'statically' referencable through the public API. * [ ]  Remove Tarpaulin format, and add new format for Cobertura. * [ ]  Test uploading reports using Istanbul and Tarpaulin.
kjhoerr added the
enhancement
label 1 year ago
kjhoerr added this to the SUI Feature Incubator milestone 1 year ago
kjhoerr commented 1 year ago
Poster
Owner

Cobertura XML is the desired format - it is the default XML format for Tarpaulin, and also has support from Istanbul and other major code coverage tools. It seems like a good default format to support.

Cobertura XML is the desired format - it is the default XML format for Tarpaulin, and also has support from Istanbul and other major code coverage tools. It seems like a good default format to support.
kjhoerr commented 1 year ago
Poster
Owner

This is going to be my main development focus for the time being. Here's a breakdown of the changes to be made:

  • First phase (this issue):
    • Replace POST API endpoint for HTML with XML. Functionally very similar, since the XML file will also be 'statically' referencable through the public API.
    • Remove Tarpaulin format, and add new format for Cobertura.
    • Test uploading reports using Istanbul and Tarpaulin.
  • Second phase:
    • Add a new template HTML page to display the XML report.
    • Additional static files as needed (icons, stylesheets, scripts, etc.)

There shouldn't need to be any additional metadata stored this way. One of the main reasons for going this direction is that the report display page should be able to be updated without retroactive processing. Keeping a template to generate should simplify things, and if any extra metadata is needed for the template, defaults can be used or figured through some migration.

This will still cause a roadblock when integrating other formats. There is still a singular/primary format, and it is not being extensively parsed for metadata (or more accurately, just data). Hypothetically, this could be eventually converted to some JSON file on the backend, and the XML endpoint dropped. That would be the third phase: to support more formats. I think Cobertura covers plenty of cases though, and for SUI purposes it will work fine for now. Different issue, different time.

If or when another format gets added, there should be some metadata added to track the original upload format. That way the original format report file can still be referenced.

It would be neat to someday have a secondary process running to batch format conversions. It would reduce processing time at upload for the server so the server could focus on serving and handling reports. As a bonus, this could nullify need for a migration (aside from updates to the intermediary format itself).

This is going to be my main development focus for the time being. Here's a breakdown of the changes to be made: * First phase (this issue): * Replace POST API endpoint for HTML with XML. Functionally very similar, since the XML file will also be 'statically' referencable through the public API. * Remove Tarpaulin format, and add new format for Cobertura. * Test uploading reports using Istanbul and Tarpaulin. * Second phase: * Add a new template HTML page to display the XML report. * Additional static files as needed (icons, stylesheets, scripts, etc.) There shouldn't need to be any additional metadata stored this way. One of the main reasons for going this direction is that the report display page should be able to be updated without retroactive processing. Keeping a template to generate should simplify things, and if any extra metadata is needed for the template, defaults can be used or figured through some migration. This will still cause a roadblock when integrating other formats. There is still a singular/primary format, and it is not being extensively parsed for metadata (or more accurately, just data). Hypothetically, this could be eventually converted to some JSON file on the backend, and the XML endpoint dropped. That would be the third phase: to support more formats. I think Cobertura covers plenty of cases though, and for SUI purposes it will work fine for now. Different issue, different time. If or when another format gets added, there should be some metadata added to track the original upload format. That way the original format report file can still be referenced. It would be neat to someday have a secondary process running to batch format conversions. It would reduce processing time at upload for the server so the server could focus on serving and handling reports. As a bonus, this could nullify need for a migration (aside from updates to the intermediary format itself).
kjhoerr modified the milestone from SUI Feature Incubator to v0.5.0 1 year ago
kjhoerr added a new dependency 1 year ago
Sign in to join this conversation.
No Label
bug
No Milestone
No Assignees
1 Participants
Notifications
Due Date

No due date set.

Blocks
#13 Create template for XML reports
kjhoerr/ao-coverage
Loading…
There is no content yet.