Problem/motivation
User Story
As a contrib developer,
I want to have the option to use semver release tags like 7.x-1.0.1 for my modules,
So that I can communicate to users the difference between releases that change features vs bugfix/security releases
Why now?
The D.O Engineering team is currently working on getting a composer shim service in place. It makes sense to tackle this at the same time.
Background
#1612910: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc) - There is a 300+ comment ticket as a policy proposal to do this in contrib. There have been many proposals debated and there seems to be some consensus around allowing a 7.x-1.0.1 format for contrib releases and because it should be the easiest option to enable on d.o -( per @drumm comment )
Usage examples
I have a release that will only contain bugfixes - A bunch of bugs have been fixed, but there are no functional changes in the code. As a developer, I want to signify this fact by only incrementing the patch position in the semver version identifier.
I created an "oops" release - As a developer, I push a new 7.x-1.0 release of my module and then someone comes along letting me know that I made a small typo and my module is causing errors. I'd prefer to release a 7.x-1.0.1 to signify that this release was just to fix an small issue and doesn't include new features.
I fixed a security issue - As a developer, the security team find a security issue that needs to be resolved in my 7.x-1.0 release, but there are also new features in my 7.x-1.x branch that are not ready for release just yet. Instead of pushing all this code out at once, I can just create a new branch from 7.x-1.0, patch the security issue, and then release a 7.x-1.0.1 branch. (in addition to patching 7.x-1.x) so that users can upgrade with more confidence knowing that the only changes were minor / security patches.
I host a distribution / install profile on d.o - As a distro developer, I use drupal.org.make files and my distro is on release 7.x-1.4. Drupal or some set of the modules that my distro relies on have security releases before I'm ready to release a new version of my distro. I want to create a 7.x-1.4.1 release so that I can update just my make file for 7.x-1.4 to use the latest drupal and module security releases.
Proposed resolution
- API-compatibility-prefixed releases will remain mostly same,
{api compatibility}-{major}.{minor/patch}. We currently store the last position as “patch”, but treat it more like “minor” in places like the Composer façade. - If you omit the API compatibility prefix, tagged releases will be semver
{major}.{minor}.{patch}. - Dev release branches may be
{major}.xor{major}.{minor}.x. If a project intends to do releases linearly, the maintainer can omit maintaining a branch for each minor version.
Remaining tasks
Ensure Drupal.org services support
#3104693: Get packages.drupal.org ready for semver
#3091062: Package translations for semantically-versioned contrib releases
#3069715: ftp.drupal.org has fallback regex's that need to account for semver.
✨Enabling semantic versioning for all projects✨
- Semantic version releases for contrib should not available until Drupal fully supports this. All of the must have issue but first completed here #3009338: [META] Support semantic versioning for extensions (modules, themes, etc) in Drupal core, and allow modules to be compatible with Drupal 8 and 9 at the same time
- Final enabling for all projects will be
drush vset project_release_version_format_project_release '!api#major%minor%patch#extra', then clean up any projects that were used for testing,SELECT * FROM field_data_field_release_version_format;
Documentation & UI updates
#2297975: Update docs to support semantic versioning
#3115728: Project Documentation forms need to allow for semver.
Cleanup old data
#3114968: Clean up now-unused columns
#3104698: Remove {project_release_supported_versions}.tid column
#3104888: Remove 9.x API compatibility term, if possible
User interface changes
Should be done soon, not a blocker to enabling for all projects.
#3104886: Update/remove API version filter on “All releases” pages
#3089291: Group issue version “series” by branch instead of API compatibility
API changes
Completed as part of #2688479: Update update status XML for Drupal 9 and contrib semantic versioning, documented at https://www.drupal.org/drupalorg/docs/apis/update-status-xml
Data model changes
Wherever version identifiers are stored, three positions should be allowed.
| Comment | File | Size | Author |
|---|---|---|---|
| #33 | Screen Shot 2020-03-31 at 9.50.15 pm.png | 95.46 KB | dpi |
| #13 | 2681459-draft.patch | 1.56 KB | drumm |
Comments
Comment #2
frankcarey commentedComment #3
frankcarey commentedComment #4
frankcarey commentedComment #5
eelkeblokApplied issue template.
Comment #6
eelkeblokNote that the patch position wouldn't exclusively be for smallish non-feature releases; it would be for any release that only contains bugfixes and no new features. I updated the usage examples.
Comment #7
eelkeblokComment #8
skaughtany motion for this?
I'm currently in this situation where i can't really push a minor release and simply i should be able to use 8.x-1.0.1 as a tag!
Release naming conventions
Comment #9
frankcarey commentedFYI: I'm proposing a 2 phased approach in #1612910: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc)
- Phase 1 - Support an optional 7.x-a.b.c tag (this ticket)
- Phase 2 - Figure out what to do with the drupal version part of the tag (7.x, 8.x)
See: https://www.drupal.org/node/1612910?page=1#comment-11377843
Comment #10
MixologicPostponing per #1612910: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc)
Comment #11
drummAdding parent and related issue from #3054391: [META] Support semantic versioning on drupal.org for contributed projects
Comment #12
drummComment #13
drummUpdating the proposed resolution to what Mixologic & I have been discussing, and attaching a draft of part of the change.
This is coming up now because modules & themes can be compatible with both Drupal 8 and 9, and the API compatibility prefix is no longer a useful requirement.
I’m currently working on #3087220: Migrate project_release_supported_versions to store “branch” instead of API/Major, and #3085041: Drupal core 9.x.x releases should not attach API compatibility term is helping us actually remove our dependence on API compatibility terms. Once some of the background work is done, a test contrib project that uses semantic versioning will likely be the next test of everything.
Comment #14
tedbowRelating 2 issues
#3074993: Use "current" in update URLs instead of CORE_COMPATIBILITY to retrieve all future updates and then after that #3100115: The update module should recommend updates based on supported_branches rather than major versions majors will happen.
I don't think we should allow the current issue to be implemented until both those are done because otherwise the core update module won't be able to handle the new releases.
Comment #15
tedbowIs there going to be restriction on create releases/branches for existing projects?
For instance if if there is an existing project that has a 8.x-1.x branch with a 8.x-1.0 release when have semantic versioning tag/releases for contrib will they have to jump to the next major?
So are would prevent the above project from making a 1.0.0 release?
Comment #16
drumm#15 - Yes, we will need to implement that restriction. Since Drupal.org’s Composer endpoint already translates 8.x-x.y version numbers into semver, we’ll be treating the new semantic version numbers as a continuation.
Comment #17
tedbowUpdate issue summary to reflect that issues mentioned here must be finished #3009338: [META] Support semantic versioning for extensions (modules, themes, etc) in Drupal core, and allow modules to be compatible with Drupal 8 and 9 at the same time
Comment #18
drummThe child issues are now reasonably complete for the known work to do. #3104912: Swap storage of version minor & patch is the next issue in the critical path. Then #2189131: Update DrupalorgVersioncontrolLabelVersionMapperGit for semantic versioning in contrib to allow making a test project with semver releases to exercise everything for real. The rest have some interdependencies, but mostly don’t block each other.
Comment #19
drummComment #20
drummAdd note to issue summary on how to enable this for all projects toward the end of the process.
Comment #21
drummThe key work here is done, we can now selectively allow projects to have semantically-versioned releases. https://www.drupal.org/project/semver_example is an example. This lets us test everywhere these releases are processed & displayed.
For core, #3100115: The update module should recommend updates based on supported_branches rather than major versions majors can have more confidence in writing test fixtures, since https://updates.drupal.org/release-history/semver_example/current has been updated.
For Drupal.org, the next priority is getting the semver releases installable via Composer, #3104693: Get packages.drupal.org ready for semver. Once that is done, making sure localize.drupal.org is working well, #3091062: Package translations for semantically-versioned contrib releases & #3069715: ftp.drupal.org has fallback regex's that need to account for semver. are high on the list. And of course, more cosmetic issues like #3111408: Make sure semver releases can show on project page.
Once there is enough support, we can open up semantic versioning to all contrib projects. With clear documentation that semantically-versioned releases will not work as well before Drupal 9, 8.9, or however far support can be back-ported.
Comment #22
pasqualleWhere should I see the semantic version with the semver_example project? Is it on admin UI only?
Comment #23
pingwin4eg@Pasqualle view all releases.
Comment #24
Mixologicadding nooks and crannies for semver support.
Comment #25
drummUpdating the issue summary to categorize the remaining child issues.
We’re pretty close to being capable of enabling this for all projects. The semver_example module is installable via Composer, #3104693: Get packages.drupal.org ready for semver remains open for some double checking on edge cases. #3091062: Package translations for semantically-versioned contrib releases is working well, likely only needs some final UI work put in place.
Comment #26
dpiI'm currently holding projects I managed from back from D9 support pending semver compat.
What can I do to opt-in and test work to date?
I'd like to be able to provide advice and direction to consumers, and project dependents.
If this is the place, projects I'd like to 🐹 guinea pig are date_recur + smsframework.
Good work guys, I'm on the edge of my seat ;)
Comment #27
drummFrom a Drupal.org perspective, we are ready to enable for all projects. I think Drupal core may be ready after the recent releases.
Last week, I updated https://www.drupal.org/node/1015226#semver-transition to explain the transition. I think the key part to communicate will be:
And I need help filling out:
That said, I think it is okay to enable for these two projects, and https://www.drupal.org/project/twig_tweak which will have D9-only releases.
Comment #28
drummI confirmed with core maintainers this is not ready for all projects, #3009338: [META] Support semantic versioning for extensions (modules, themes, etc) in Drupal core, and allow modules to be compatible with Drupal 8 and 9 at the same time still has a must-have issue that needs to be completed, and some issues need triage.
That is provisionally Drupal 8.8.3.
These projects can now make releases with semantic versioning:
Comment #29
wim leersAs of https://www.drupal.org/project/cdn/releases/8.x-3.4, the CDN module only supports Drupal >=8.8. (https://www.drupal.org/project/cdn/releases/8.x-3.3 supports Drupal 8.7.x too.)
Note that it also still has a Drupal 7 branch that is supported, with the latest release being https://www.drupal.org/project/cdn/releases/7.x-2.10.
I'm happy to also be a guinea pig with the CDN module if you are looking for more testers. There already is an issue for a
4.xversion with an outline of what would be different with the previous major version: #3103682: CDN module 4.x: no new features + drop BC layers + drop now irrelevant headers + PHP >=8.1 + Drupal >=9.4.Comment #30
drummhttps://www.drupal.org/project/cdn can now make releases with semantic versioning.
With #3089291: Group issue version “series” by branch instead of API compatibility now done, there is currently only one minor UI update for Drupal.org remaining, #3115728: Project Documentation forms need to allow for semver.. The rest is cleanup that no one will notice, if it goes well.
Core still needs some work for support: #3009338: [META] Support semantic versioning for extensions (modules, themes, etc) in Drupal core, and allow modules to be compatible with Drupal 8 and 9 at the same time
Comment #31
chi commentedI have just published the first semantic release of Twig Tweak. Everything works well. You have done great work. Thanks.
Comment #32
andypostWorks for me https://www.drupal.org/project/twig_tweak/releases/3.0.0-beta1
Comment #33
dpiThanks @drumm, works great! First release created for https://www.drupal.org/project/date_recur.
Bring on the compatibility tables! 😬
Comment #34
markhalliwellActually, I think the majority of that could be handled automatically by #3121281: Add Drupal core compatibility info to release & project pages.
Comment #37
drummThis is now enabled for all projects! Details on how to number your next version are at https://www.drupal.org/node/1015226#semver-transition. The short version is:
8.x-3.5, your next stable release will be4.0.0There are still core issues relating to semantic versioning, since places like update status now have multiple upgrade paths to consider. Help fix those issues at #3009338: [META] Support semantic versioning for extensions (modules, themes, etc) in Drupal core, and allow modules to be compatible with Drupal 8 and 9 at the same time
These three projects are using semantic versioning now, and are good examples for helping with those core issues:
Comment #38
chi commentedCool. I think the CR needs to be updated and published.
Comment #39
drummI forgot I wrote that. Done: https://www.drupal.org/node/3108648
Comment #40
geek-merlinGreat stuff. I tried this for og_members_tab, but i can't set my 1.x dev version to supported(here). Is this a bug or a voluntary but undocumented change?
Comment #41
tr commented@geek-merlin: See #3125239: Suggest 1.0.x as the first branch to create instead of 1.x and this thread: https://drupal.slack.com/archives/C1BMUQ9U6/p1586963356133500
Comment #42
geek-merlin@TR Thanx a bunch!
Comment #43
ccjjmartin commentedComment #44
drummMany releases have been made using semantic versioning, and I’ve completed the cleanup issues. We can call this done now!
Comment #45
kim.pepperWoohoo! Thanks!