Dev environment setup for just compiling

[darrenmason]

Hi all,

Has someone recently setup a dev environment for 810 that can assist with this question?

I assumed I could just setup an environment for just compiling the current SVN codebase. (ie: Just code compile, no packaging/ CD creation/ DB build etc). This was certainly previously possible but has this now changed?

I have had a go at it but had some problems with unmet dependancies. I have just found the list of 'build-packages' so that might solve that when I get home.

I was just questioning on whether this is how any of the developers on 810 were working or are you all building everything as per the build scripts (and wiki instructions)

Regards
Darren

[tschak909]

 Darren,

We all have our own build servers set up as per the "Building Linux 8.10 from source." instructions.

Myself, I have one clean checkout in /home/src/

and then there is the one in the builder chroot.

I hack on the one in the builder chroot, use test-pkg.sh to build the package, test...hack....build...test....repeat...

then copy the changed code into the clean co in /home/src to commit.

This way the prepped makefiles are not checked in.

It will be this way for the foreseeable future until the prepping is removed, after the 0810 release.

Now, for DCE device development, DCE devices are usually simple enough to do dev work outside the builder, check it in, and then use the builder to verify that packages are being correctly built.

I used the 0710 prepping instructions to make scripts called prep.sh and unprep.sh which prep and unprep for new DCE driver development.

-Thom

[LegoGT]

I've only recently started getting the hang of the full source build environment, so take my suggestions with a grain of salt. One of my main issues was that I work from home and can't have my main network tied up or torn down when re-installing or rebooting LMCE. I also wanted to reduce the craziness of my network cabling scheme.

After much frustration, I finally decided to virtualize the LMCE core machine. I already had an internal test server with VirtualBox and can now use it to keep "snapshots" of my system at certain configuration points. Also, this lets me run a separate VM running just the build environment in parallel.

I use VirtualBox on an Ubuntu Server 9.04 machine with 2 NICs. One points towards the always working network (cable modem) and the other points towards my LMCE test network -- both have WiFi routers. My server, LMCE VM, and builder VM all straddle both NICs using bridged interfaces (so they all get their own "true" MACs and IPs). To help out, I set up VBox to run headless and can start and stop each one using init scripts. Logging in is easy over SSH and even using RDP to peek at the screens!

So far, I haven't had any trouble using N800s, Media Directors (1080p), my HDHomeRun, Squeezecenter, and even USB devices (ZWave, DIO boards, and RS-232 for GSD).

I know this isn't for everyone but it might ease the pains of dedicating separate machines to test, run, and build LMCE. Until I get everything sorted out it keeps media management between rebuilds a little easier, too (music and DVDs stay shared on the server for LMCE to "find" on each re-install).

Finally, I followed the same "Building 8.10 from Source" Wiki article to set the builder VM up... very helpful! You might want to also refer to this post from Thom (http://forum.linuxmce.org/index.php?topic=7462.msg55175#msg55175) for additional help on an effective build process.

[sp00nhead]

How big is your VM? If its small enought we could look at having a semi offical VM dev image and share it on bittorrent, would help get those without a dev box start to get involved.

Could you pop a walk through on the wiki on how your git it from zero to ready to dev with?

[wierdbeard65]

That would be awsome!

Obviously, once you've downloaded it, you'd have to update it to the latest build (unless someone decides that part of the build is to update the VM image  )

Before anybody goes through the pain and puts in a lot of work, however, it might be worth conducting a poll of which virtualization software folks use / are comfortable with. As I have to use Windoze on my laptop for work, I would need to run it on software that supports that as a host OS. I currently use VMWare, but am happy to change if there's a better alternative. I would prefer not to go down the whole dual-booting route if I can avoid it and I'm not sure my laptop is up to running Windoze as a VM for my day-to-day work (although it's a thought!)

Just my 10 Eurocents

[MarcoZan]

Maybe a dumb question ....

Currently I have no hardware to dedicate to a test environment (I have only my current production LMCE 7.10 core, and a diskless MD, no other hardware), so I'm also thinking towards VM.

My concerns are related to network config. Core is designed to work with 192.168.80.1, so I'm afraid that as soon I fire up my 8.10 test  VM when connected to my home network suddenly my production 7.10 could start complaining.

Are there suggestion on this?

Thx
Marco

[sp00nhead]

You can Set teh VM internal network not to be shared on your network, so it only stays on your dev machine, Or you can change to DHCP range of teh dev box to something like 192.168.81.xxx

With VM machine you can also do cool stuff like linking teh virtual network directly to another local virtual machine, so you can netboot a test MD on the same box.

[MarcoZan]

Thanks spoonhead for your reply

I've always heard bad stories about changing the core default network settings, so I prefer to stick with them and possibly work on sharing/unsharing VM network.

Sounds like a VM with 8.10 test bed on my laptop is in my near future ...

Thanks again
Marco

[merkur2k]

virtualbox at least has an option for using "NAT" (in fact its the default) for networking, which sidesteps the problem completely. other vm software should have the same option.

[MarcoZan]

Thanks merkur2k

this means that I could also dare to put a test VM directly on my production core, am I right?

Marco

[sp00nhead]

If you don't want to risk your main core, i'd stick with using your laptop for dev.
just on the safe side
But what should work is if on the core,  your internal nic could be bound to either your real core or your Vm core. Now i've not looked at what tools Virtualbox has, but this should be easy to do on the fly, so you could netboot your existing devices off your VM core to test.

On a side note, teh new version of Virtualbox allows openGL accelration when teh host system is linux.... my crazy brain is thinking multi-head box with a bunch of VM MDs all playing back dvds....

[darrenmason]

Darren,

We all have our own build servers set up as per the "Building Linux 8.10 from source." instructions.

Myself, I have one clean checkout in /home/src/

and then there is the one in the builder chroot.

I hack on the one in the builder chroot, use test-pkg.sh to build the package, test...hack....build...test....repeat...

then copy the changed code into the clean co in /home/src to commit.

This way the prepped makefiles are not checked in.

It will be this way for the foreseeable future until the prepping is removed, after the 0810 release.

Now, for DCE device development, DCE devices are usually simple enough to do dev work outside the builder, check it in, and then use the builder to verify that packages are being correctly built.

I used the 0710 prepping instructions to make scripts called prep.sh and unprep.sh which prep and unprep for new DCE driver development.

-Thom

Thanks Thom - will try and replicate this on the weekend. Actually solved the dependencies not long after I posted but since I need to rebuild DCERouter to account for the new plugin I think I need to replicate the full build environment.
(I actually get DCERouter to compile with my new plugin but executing complains about not finding MSQL Wrapper)

[darrenmason]

Thom (or anyone),

Still missing something I think.

Done a 'chroot' build exactly as described. No Major dramas - just took a while.

Now have 'var/www' populated with the packages as expected.

On the core(normal - not chroot) I did the following 'ln -s /media/kubuntu-810/var/www /var/www/buildrepo'
This gives me a browserable view of the directory.

Added line to sources.list for core
'deb http://localhost/ ./buildrepo/'

apt-get update doesn't complain but I get a number of Ign lines - which I took to mean Ignoring this repo as it is up to date/no changes etc. Am I right here?

Unsure of what the path/steps are now to move along.

I was assuming that putting the new sources.list entry in would mean that when I run 'apt-get upgrade' I get the linuxmce packages refreshed from my repo (since all of them should have very recent timestamps). Instead I am getting 'everything is up to date'.
This leads me to believe that embedded package versions are used rather than just timestamps. Is this correct?
Should I just assume that the packages I have built are exactly the same as the ones that I have got installed and just get on with it or do I need a way to make it use the packages from my repo?

Also assumed 'test-pkg.sh' basically will just rebuild and package a single package. Is this correct?

I am guessing that it will increment a version or do something that will cause the package to be reinstalled. Correct?

Will try a few of these things when I get home this evening - just trying to avoid some trial/error pain if I can....

Thanks
Darren

[colinjones]

I do know that deb uses fairly sophisticated package numbering, so when you pull down an update, it updates your local deb database cache with the entire version number dependency hierarchy, so that when you do an upgrade it pulls the latest version of each installed package then walks the dependency tree and compiles a list of all the versions that the package is dependent on then tries to install those first, and so on and so on. It isn't just dependency links, there are also recommended and suggested hierarchies as well... so yes, it is much more than just modified dates (in fact I don't think they play a part in any way).

Dunno why it didn't work for you, unless your ./buildrepo/ wasn't first in the sources.list file... I think it needs to be first to ensure it takes precedence, and if there are conflicting sources (in terms of latest versions and dependencies) then perhaps you need to comment out all the other repos (or at least the LMCE ones)?

Just a thought, are you sure you have a valid repo in that directory? It needs more than just the package files, it also needs the deb meta data stuff that describes the dependency schema and actual data. Can't remember what it looks like, but I know I browsed a repo recently, found the meta data files, and then combed through them to find the specific file paths within that repo to get a particular package ... from memory it was right near the top of the folder structure and off the "left" somewhere in another folder!

[merkur2k]

The metadata info that you speak of is kept in a Packages file (and also compressed as Packages.gz). One of the build scripts creates this, so you can see how it is done.