Author Topic: Slow and steady ... moving from Trac to Gitlab  (Read 20672 times)

polly

  • Administrator
  • Guru
  • *****
  • Posts: 209
    • View Profile
Slow and steady ... moving from Trac to Gitlab
« on: August 10, 2015, 12:09:14 pm »
Hey ...

just wanted to let you guys know that i'm done with "trac to gitlab" and "svn to git" migration.
There is already a cronJob running once a day at 8:30am CEST pulling all data over from trac and svn.

From now on, testing begins! :-)
Let me know if you found anything weird or missing tickets and notes/changelogs.

For most users i was able to map their trac usernames to gitlab users.
I tried to merge the users as much as i could. So if i found a user in trac which had the same name in the forum, but a different e-mail i went with the forum account.

If you are not sure if/which of your accounts is still there use the forgot password feature in Gitlab.

For now Gitlab is READ ONLY! ... this will be changed as soon as everybody is happy and comfy with GIT.

LinuxMCE Gitlab installation:
http://git.linuxmce.org

Gitlab Android App:
https://play.google.com/store/apps/details?id=com.bd.gitlab

Gitlab iPhone App:
https://itunes.apple.com/us/app/gitlab-control/id654747119?mt=8

Roadmap:
From now on, CronJob will update tickets and repos once a day. then we have at least 4 weeks of testing and playing. After that the devs decide when switch completely to git.

Let me know if you have any issues!

Thanks.

Cheers,
ochorocho
« Last Edit: August 10, 2015, 07:56:36 pm by polly »

mkbrown69

  • Guru
  • ****
  • Posts: 213
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #1 on: August 12, 2015, 05:58:16 am »
Sweeet!  And thanks for your efforts! 

At work, we switched from Git/Trac to Git/Gitlab, and it's sooo much more of a productive environment. We've got hooks from certain projects into Jenkins for building RPM's 5 minutes after a commit, and that's sweet from a packaging point of view. Jenkins really rocks as well!  (BTW, there's a .deb builder plugin for Jenkins if that's of interest to you)

I have a small request of you and the core devs.  Gitlab represents a real opportunity to enable more people to get involved in LinuxMCE development and testing. In the past, I didn't want to have any form of SVN access;  I know enough to know I don't know enough about LMCE to event want any commit access (bull in a china shop syndrome). So, that limited me to filing tickets with diffs attached.

Gitlab changes all of that. The core devs can steward the main linuxmce project, and personal workspaces allow more novice folks like myself the opportunity to contribute through an isolated environment and merge requests. So, I would like to make the following request:

Could you folks create a wiki page with some example workflows that would enable folks like myself to contribute more, while minimizing the level of effort on the core devs to take advantage of what we do?

I'm thinking of example workflows like:
  • simple bug fix for existing code
  • significant bug fix for existing code or refactoring
  • simple new feature (new code)
  • significant new feature (majorly intrusive)

I think that giving us examples of how you want us to work would be good for us novices to learn, while making life easier for those already overcommitted core devs.   Maybe even a quick page on how issues should be filed might be good for standardizing and leveraging Gitlab's capabilities...

Just thought I'd put this out there for your consideration.

Thanks again for your hard work on this!  I'm sure it wasn't easy, but it will be worth it in the long run!

/Mike

polly

  • Administrator
  • Guru
  • *****
  • Posts: 209
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #2 on: August 12, 2015, 12:26:18 pm »
Hey Mike,

thanks for writing this detailes message.
I agree, me as a semi skilled programmer likes the idea of fork/pull request.
So core devs review the code i wrote i give us a "well done" by accepting the merge request.

The website is a good place to put info about contributing to LMCE: http://www.linuxmce.org/contribute.html

Cheers,
ochorocho

polly

  • Administrator
  • Guru
  • *****
  • Posts: 209
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #3 on: August 20, 2015, 08:33:25 am »
No issues at all? ... can't be! ;-)

phenigma

  • LinuxMCE God
  • ****
  • Posts: 1758
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #4 on: August 20, 2015, 08:59:43 pm »
Haven't been testing it yet.  Soon...  You'll have 1000 tickets to act on when you get back ;)

J.

polly

  • Administrator
  • Guru
  • *****
  • Posts: 209
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #5 on: August 21, 2015, 05:06:33 am »
Haven't been testing it yet.  Soon...  You'll have 1000 tickets to act on when you get back ;)

J.

Jiiiha! .... ;-)

tschak909

  • LinuxMCE God
  • ****
  • Posts: 5549
  • DOES work for LinuxMCE.
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #6 on: August 22, 2015, 04:33:41 am »
Sweeet!  And thanks for your efforts! 

At work, we switched from Git/Trac to Git/Gitlab, and it's sooo much more of a productive environment. We've got hooks from certain projects into Jenkins for building RPM's 5 minutes after a commit, and that's sweet from a packaging point of view. Jenkins really rocks as well!  (BTW, there's a .deb builder plugin for Jenkins if that's of interest to you)

I have a small request of you and the core devs.  Gitlab represents a real opportunity to enable more people to get involved in LinuxMCE development and testing. In the past, I didn't want to have any form of SVN access;  I know enough to know I don't know enough about LMCE to event want any commit access (bull in a china shop syndrome). So, that limited me to filing tickets with diffs attached.

Gitlab changes all of that. The core devs can steward the main linuxmce project, and personal workspaces allow more novice folks like myself the opportunity to contribute through an isolated environment and merge requests. So, I would like to make the following request:

Could you folks create a wiki page with some example workflows that would enable folks like myself to contribute more, while minimizing the level of effort on the core devs to take advantage of what we do?

I'm thinking of example workflows like:
  • simple bug fix for existing code
  • significant bug fix for existing code or refactoring
  • simple new feature (new code)
  • significant new feature (majorly intrusive)

I think that giving us examples of how you want us to work would be good for us novices to learn, while making life easier for those already overcommitted core devs.   Maybe even a quick page on how issues should be filed might be good for standardizing and leveraging Gitlab's capabilities...

Just thought I'd put this out there for your consideration.

Thanks again for your hard work on this!  I'm sure it wasn't easy, but it will be worth it in the long run!

/Mike

A good post! Of course ,you're assuming we actually _have_ a workflow, but we don't....really....other than:

* File a ticket in trac
* assign your commits with Refs #1234 for that ticket
* make sure when you're done with it to close it, either with Fixes #1234, or by marking it closed or fixed. (depends on the ticket type).

Maybe we should... I dunno, what do you think, posde?

-Thom

phenigma

  • LinuxMCE God
  • ****
  • Posts: 1758
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #7 on: August 26, 2015, 06:14:08 am »
I think this is a great idea... once I understand step one of git.   :o  I think we are still at the initial goal of getting most of svn migrated (at least the current stuff).  I'm not sure yet if any one of us has even tried to clone the git repo...  heh.  That aside I am setting up a new builder to try and migrate our existing build scripts from svn to git.  mkbrown perhaps you want to get involved a little bit more at the early stages here and take the lead on developing a workflow wiki page for git.  I for one could benefit from any additional knowledge I can get my hands on.

J.

mkbrown69

  • Guru
  • ****
  • Posts: 213
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #8 on: August 27, 2015, 03:54:23 am »
Ok.... It could be a case of the blind leading the blind here... ;p

To develop workflows, we'll have to decide on how we're going to handle branches and tags.  I'd suggest folks take a read here: https://about.gitlab.com/2014/09/29/gitlab-flow/

The "Gitlab" flow may be what we want to consider.  Right now, we have everything dumped into "master". Maybe we want to consider master as the common code repository, and create branches for each version release? That does mean maintaining pulls across however many streams (10.04, 12.04, 14.04) we intend to maintain. Tags can be used for release candidates or versions.

Or we go with master as production ready, and put conditionals into the code for every stream supported for the builders. Develop and feature branches are where all the work gets done. Tags can still be used to mark versions, milestones, RCs, etc.  Releases get pulled over to master for the builders to crank out release packages. Develop could get nightlies. 

I think a lot of this now comes down to release planning. Are we running a continuous integration type release process, or a waterfall staged type process?

Just some points to consider...  HTH!

/Mike
« Last Edit: August 27, 2015, 04:23:38 am by mkbrown69 »

mkbrown69

  • Guru
  • ****
  • Posts: 213
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #9 on: August 27, 2015, 05:32:45 am »
A separate post to give an idea of my frames of reference. At $day_job, my role is now architecture and integration to support SysAdmins (which I used to be).  To do that, and to support a varied environment consisting of many flavours of Unix, Linux, Hypervisors and hardware architectures, we implemented a configuration and compliance management tool known as Puppet.  Puppet uses it's own structured language to describe system end-states, and maintain them as a configuration and compliance engine.  So, basically, you have infrastructure as code.

So, when you have code, and a need to maintain transparency and sustainable, reproducible infrastructure, you need a version control system. Originally, that was Git and Trac for us, and we made the switch to GitLab a few months ago for better productivity and for integration into other tools like Jenkins. Because, when you deal with almost every form of *nix out there, keeping your infrastructure tooling consistent across all platforms is challenging. You can't always ensure that the vendors will be shipping exactly what you need, and so you can be dealing with even more variety in your tooling as well as the stuff your're trying to maintain! It's enough to make you want to smash something!  So, sometimes you need to roll it yourself in order to ensure you have what you need, just the way you need it, when you need it. So, we use Jenkins for building packages to ensure our infrastructure tooling is in lockstep across all platforms.

So, basically I'm a SysAdmin using GitLab to maintain the tools my team needs to manage hundreds of systems. Our Git workflow represents a fairly linear model as a result.  We're just always making releases to increase capabilities or service catalog products, or just to deal with the fact that things change (like System V init to systemd). As such, we maintain a fairly simple workflow.

Master is always production-ready. Features get developed in their own branches. The develop brand is where integration happens, and dedicated "life-cycle systems management" instances are subscribed to that branch for dev/test and regression testing. Releases are tagged, and pulled into master.  Master gets pulled into the systems management tool, and applied to client test/dev systems. Upon acceptance from them, they get applied to validation systems. If those are good, it gets air-gapped across to client staging/pre-prod instances and tested against a back-flush of prod.  Assuming no regressions, it gets promoted to Production systems. Rinse, repeat for next cycle.

So, in a lot of ways, workflow gets determined by how much risk someone is willing to accept. In a high availability environment, very little risk is accepted; hence a rigorous process with many gates and go/no-go decision points. Tooling always has to support the process. More risk tolerance lends itself to more of a continuous-integration type workflow, where small changes are integrated often and distributed widely, and the tooling usually supports a quick reversion capability if things go badly.

Most of the opensource projects that get used in the enterprise space operate on a two-pronged approach. Nightly builds for those who develop or who like living on the edge. Stable releases are for those who have to depend on it.

So, do we want to model or mimic workflows of similar projects (some of whom are on GitHub)?  The fork, branch, pull-request model does work well for most of those projects. We just need to consider the other tooling (like the builders)  and the overall release process for getting it out into the wild. However, LinuxMCE has challenges that something like OpenHab or AgoControl won't have. LMCE's "auto-magic" functionality requires deep hooks into operating system components, and those are OS version specific. So our processes and workflows may need to consider and allow for supporting an 'n', 'n+1', and 'n-1' release strategy.  Branching may or may not reflect that strategy, depending on how other processes and tooling handle it.

We have community contributors, and there's also Dianemo/CHT. Do we treat them any different from the community?  GitLab allows for personal workspaces and team workspaces... GitLab also allows for branches to be "protected", limiting those who can commit to it. This is a good idea for production branches.

More food for thought. Opportunities to cast aside legacy, and the cost is to consider all the options, and possibly blaze a new trail.

/Mike

Marie.O

  • Administrator
  • LinuxMCE God
  • *****
  • Posts: 3676
  • Wastes Life On LinuxMCE Since 2007
    • View Profile
    • My Home
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #10 on: August 27, 2015, 08:21:56 am »
A note on CHT/Dianemo: They have long used their own repository, but do provide fixes back to us every now and then manually.

polly

  • Administrator
  • Guru
  • *****
  • Posts: 209
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #11 on: January 02, 2016, 02:05:29 pm »
Now its time for testing! ...
Let me know if anything does not work as expected.

Url: https://git.linuxmce.org/
If you ever made a SVN commit you should have received a mail containing a reset password link, it may be marked as SPAM/Junk.

i already tested following:
- Checked issues and compared with trac … like every 100th issue — CHECK
- Checked commit history back to 2010 — CHECK
- Issue overview: checked Filter — CHECK
- Git clone — CHECK
- Git commit — CHECK
- Graph, works except for Languages, maybe this could not converted — CHECK
- SSH Key — CHECK
- Milestones page complete incl. progress bar — CHECK
- E-Mail notifications work, but may end up in Junk/SPAM folder — CHECK
- SSL — A rating should be fine: https://www.ssllabs.com/ssltest/analyze.html?d=git.linuxmce.org — CHECK -- due to proxy issues during login i disabled HTTPS for now....

if you have further questions im on irc:ochorocho

Thanks.
« Last Edit: January 02, 2016, 09:59:31 pm by polly »

phenigma

  • LinuxMCE God
  • ****
  • Posts: 1758
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #12 on: January 02, 2016, 08:13:19 pm »
Nothing is working as expected  :P  see issues in irc.

J.

polly

  • Administrator
  • Guru
  • *****
  • Posts: 209
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #13 on: January 02, 2016, 09:58:07 pm »
 :o

For now HTTPS is disabled...
http://git.linuxmce.org/

and it should work now.

polly

  • Administrator
  • Guru
  • *****
  • Posts: 209
    • View Profile
Re: Slow and steady ... moving from Trac to Gitlab
« Reply #14 on: January 03, 2016, 08:54:11 pm »
Yey! .... https is working again:
 
  https://git.linuxmce.org/

People dir is missing, i'm currently working on this.