summaryrefslogtreecommitdiffstats
path: root/tests/baseline/shared/qbaselinetest.cpp
Commit message (Collapse)AuthorAgeFilesLines
* BaselineProtocol: add FinalizeTesting commandJonas Karlsson2024-11-211-0/+17
| | | | | | | | | | Also adds a helper function QBaselineTest::finalizeAndDisconnect() that calls the finalize command, prints the optional report URL and disconnects to the server. Pick-to: 6.8 Change-Id: Iabdba77f9eebad3e3f3fa67ab8977d1e49c49fb4 Reviewed-by: Eirik Aavitsland <[email protected]>
* Add a few improvements to the baseline testing frameworkEirik Aavitsland2024-06-101-4/+23
| | | | | | | | | | | | | 1) Add a QBASELINETEST_MAIN macro as a replacement for QTEST_MAIN, relieving the clients of reproducing the kludgy workaround 2) Add a -server command line option to baseline tests, as an alternative to specifying the server in an environment variable 3) Fix command line parsing so that it exits on syntax errors. Change-Id: I36f38267143a308e971e2e7b2fdbe4be44370043 Reviewed-by: Volker Hilsheimer <[email protected]>
* Baseline tests: avoid rendering items that are blacklisted on the serverEirik Aavitsland2024-02-151-9/+22
| | | | | | | | | | | | | | | | | Blacklisting items on the baseline server would avoid any mismatch failures for such items. However, they would still be rendered. That is undesirable if the rendering itself has unwanted side effects for such items, e.g. crashing the test executable. Fix by adding new functionality in the baseline testing framework to allow checking blacklisting status ahead of time, and add a new macro QBASELINE_SKIP_IF_BLACKLISTED to do just that. Add usage of that macro to the QPainter baseline test. Pick-to: 6.7 6.5 6.2 Change-Id: I35f6df8cff2c6cb985c25ab5470cd42b53d44940 Reviewed-by: Eskil Abrahamsen Blomfeldt <[email protected]>
* Modify the baseline testing framework to show new test itemsEirik Aavitsland2024-02-081-4/+11
| | | | | | | | | | | If a commit adds new baseline test items, they would not be reported to the server when doing a LanceBot test on that commit, and so not be visible in the report either. This commit fixes the client side of that issue; the server is already updated. Change-Id: I60c2958def5e7b54ddc2789cf5727edcbb1a90e1 Reviewed-by: Eskil Abrahamsen Blomfeldt <[email protected]> Reviewed-by: Jonas Karlsson <[email protected]>
* Change license for tests filesLucie Gérard2024-02-041-1/+1
| | | | | | | | | | | | According to QUIP-18 [1], all tests file should be LicenseRef-Qt-Commercial OR GPL-3.0-only [1]: https://contribute.qt-project.org/quips/18 Pick-to: 6.7 Task-number: QTBUG-121787 Change-Id: I9657df5d660820e56c96d511ea49d321c54682e8 Reviewed-by: Christian Ehrlicher <[email protected]>
* Port from container.count()/length() to size()Marc Mutz2022-10-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is semantic patch using ClangTidyTransformator: auto QtContainerClass = expr(hasType(namedDecl(hasAnyName(<classes>)))).bind(o) makeRule(cxxMemberCallExpr(on(QtContainerClass), callee(cxxMethodDecl(hasAnyName({"count", "length"), parameterCountIs(0))))), changeTo(cat(access(o, cat("size"), "()"))), cat("use 'size()' instead of 'count()/length()'")) a.k.a qt-port-to-std-compatible-api with config Scope: 'Container'. <classes> are: // sequential: "QByteArray", "QList", "QQueue", "QStack", "QString", "QVarLengthArray", "QVector", // associative: "QHash", "QMultiHash", "QMap", "QMultiMap", "QSet", // Qt has no QMultiSet Change-Id: Ibe8837be96e8d30d1846881ecd65180c1bc459af Reviewed-by: Qt CI Bot <[email protected]> Reviewed-by: Volker Hilsheimer <[email protected]>
* Use SPDX license identifiersLucie Gérard2022-05-161-27/+2
| | | | | | | | | | | | | Replace the current license disclaimer in files by a SPDX-License-Identifier. Files that have to be modified by hand are modified. License files are organized under LICENSES directory. Task-number: QTBUG-67283 Change-Id: Id880c92784c40f3bbde861c0d93f58151c18b9f1 Reviewed-by: Qt CI Bot <[email protected]> Reviewed-by: Lars Knoll <[email protected]> Reviewed-by: Jörg Bornemann <[email protected]>
* Baseline testing: Fix overriding of Project propertyEirik Aavitsland2022-03-171-15/+3
| | | | | | | | | | | Remove some kludgy, redundant and never used functionality for setting project and test case names, as it also hindered overriding those properties at runtime. Pick-to: 6.3 6.2 Change-Id: Ibef7d7d0cb5fc1e462752f2ba2db76cc088dbd48 Reviewed-by: Axel Spoerl <[email protected]> Reviewed-by: Volker Hilsheimer <[email protected]>
* Add a couple of utility functions to the baseline test frameworkEirik Aavitsland2022-01-131-2/+14
| | | | | | | | | | | | | | | | | | Adds a function for setting the baseline Project name, which selects the top level server directory and hence config settings. Also adds a function that lets the client override the server config setting for the project image keys (ItemPathKeys). This is the list of properties that will be used to determine which clients are "the same", i.e. which baseline to compare against. Overriding that is handy since it allows experimentation and customization of clients without involving changes in the project config on the baseline server. Pick-to: 6.3 6.2 Change-Id: Id3998356494a9a2cb71c009b43593d3dc1b6963a Reviewed-by: Volker Hilsheimer <[email protected]> Reviewed-by: Axel Spoerl <[email protected]>
* Update baseline testing frameworkEirik Aavitsland2021-11-241-4/+12
| | | | | | | | | | | | | | | | | | | Merge in various minor changes and fixes that have been done to the branched copy in qtquick3d, and clean out some outdated code. Mostly just coding style fixes and cleanups, but also: - adds -keeprunning command line parameter, intended for tests that by default exits early if it seems the platform is too unstable (e.g. crashing) for a meaningful testrun. - Changes behaviour for fuzzy matches, from SKIP to PASS. The (mis)use of QSKIP was done to force log output; now the output is just printed by qInfo() instead. Pick-to: 6.2 Change-Id: I46e77a94cc5b1980ac420086c2ae88dc9b84ef12 Reviewed-by: Volker Hilsheimer <[email protected]>
* Rename and restructure the baseline (lancelot) testing codeEirik Aavitsland2021-11-161-0/+423
In preparation for addition of new baseline tests, establish a new test category, "baseline". This is similar to the category "benchmarks" in that it contains tests that use the QTest framework, but conceptually are not unit tests, in contrast to those under auto/. Move the existing QPainter baseline test, tst_lancelot, into this new category, and rename it accordingly. Baseline tests use the QBaselineTest extension to QTest. Move that extension too into the tests/baseline directory, allowing the clean out of the baselineserver directory. Pick-to: 6.2 Change-Id: I1b527f5867c953b1d22be73798fcf7d1494712ea Reviewed-by: Qt CI Bot <[email protected]> Reviewed-by: Eirik Aavitsland <[email protected]> Reviewed-by: Volker Hilsheimer <[email protected]>
generated by cgit v1.2.3 (git 2.25.1) at 2025-07-17 00:04:59 +0000