LinuxMCE Forums

General => Developers => Topic started by: Armor Gnome on July 29, 2015, 03:15:27 am

Title: Link Aggregation in LMCE
Post by: Armor Gnome on July 29, 2015, 03:15:27 am
I would like to open a discussion topic on a feature I am interesting in developing and testing for LMCE.  This project is something I could tinker with and do development work on at a bench while the bulk of my system remains in storage.  Before I dive in however I would like to pose some function questions to those much more experienced than I to feel out what the usefulness of this would be in the LMCE network, ie. are performance improvements expected or is this a neat trial with no true benefit other than saying I did it.

Basic idea:  Provide manual configuration options to get network port aggregation working on the core.  A configuration on the web-admin for after install upgrades with manual port bonding or auto-magic detection scripts for installation that would assign bonds between NICs.

Limited initial support hardware:  dual pci-e NIC, quad pci-e NIC,  possible later add support for multiple single inexpensive PCI NICs

Concept:  In a PXE boot environment with all IO routing to and from the core, a 10/100/1000 is understood to be a minimum requirement.  From my understanding of LAG, the gigabit transmission is then split among all nodes pulling and requesting data at once limiting each connected node to a fraction of the 1000mb/s.  Two MDs would get 500mb/s each, four would get 250mb/s each.. etc.  By adding support for two (or more) gigabit NICs working as a bonded eth1 on our cores, network bottlenecks during peak traffic conditions could get closer to the 1000mb/s data per node after the switch.

Network boost, Minimal expense:  LAG would be beneficial between the core and the switch(s) only in my opinion and would require support for additional hardware on the core + extra ethernet cable to the switch.  Single lines from the switch out to MDs would not be needed or beneficial.

Developer/Network Architect Input:  Above is what works well on paper and depending on which article you read is either the greatest networking throughput gain ever or a lab experiment for a rookie Cisco student with no real benefit.

In our networks:

Are there peak transmission times where all MDs want data and we feel our cores cant provide it fast enough?
      Are MD boots done sequentially or all at once? 
      When streaming media from SAS on the core is IO bottlenecked?
Is LAG believed to be a feature worth testing and developing for LMCE 14.04 and later releases?
*Regarding LAG with switch redundancy:  Although it adds some complexity there is also the multiple bonded NICs going to multiple switches for redundancy in network topography.  I see less usefulness here in our networks but could also experiment with this easy enough.
Title: Re: Link Aggregation in LMCE
Post by: tschak909 on August 14, 2015, 06:56:18 pm
Dude, you've got my vote. What do you need from us?

Title: Re: Link Aggregation in LMCE
Post by: Armor Gnome on August 16, 2015, 08:01:41 am
Time allowance.

Things have been hectic at work for me lately which have spilled over into my weekends when I wanted to work on this.  Second smaller hurdle for me has been hardware.  I really missed the advice of developers and long time users years ago when selecting hardware.  I come from gaming machine builds which tend to prefer the higher watt multicore cpu which have noise and heat issues I wouldn't have with "adequate" hardware.  I'll want to completely rebuild my core and two MDs to set up this test bench.  I'll also be ordering core SSD which will ensure by bottleneck is network related and highlight improvements LAG provides.

What I will need help on once I have manually configured my ifconfig and lmce network settings is guidance and information.  I'll want to lead this project (otherwise it would be posted in feature request) but my coding is week and my understanding of lmce install order is as well. 

I know the dev channel in IRC is available, I plan to use it a lot once I know what the final config settings need to look like, and I know what the current installer does when multiple eth options.  That info will direct my questions to the devs on the best way to add it to lmce.

The options I see at this planning point would be:

During initial install screens, a eth scan is performed and the user configures ports.  This would allow 10/100 onboard ports to be assigned as eth0 (external) to avoid issues later.  I would also have auto and manual advanced settings if eth2+ is detected.

A web-admin add-on that could be used for after installation additions or corrections to setup defined port usage.

A graphical network map feature similar to av-wizard which would guide users to proper network building and tuning.  Statically assigning wireless access point IP address so their web interfaces can be accessed,etc.

I predict 1-2 months at this point.
Title: Re: Link Aggregation in LMCE
Post by: Marie.O on August 16, 2015, 02:25:44 pm
You might want to touch base with Alblasco1702 - he is currently taking care of all thing network related.
Title: Re: Link Aggregation in LMCE
Post by: Armor Gnome on August 17, 2015, 11:57:46 am
Noted, thank you.

I've been reading through some network config issues with the v12 & v14 installers and experienced some odd behaviour myself.  As my project is a future addition and his work appears to be more bug related I don't see much need in collaboration at this point but will appreciate his assistance when I get user ready.
Title: Re: Link Aggregation in LMCE
Post by: Alblasco1702 on August 17, 2015, 04:08:23 pm
Hello Armor Gnome,

Multiple networkcards are not good integrated at the moment i'm working to add it, and there are some bugs yes.
i have a core with four network cards and one of them is a wifi card with option to set mastermode and there for can be used as AP what i'm integrating as well.
i'm working to set the wifi card as network client as well and make it possible to add the core to an existing wifi network as well.
or for some s,aller thing ad-hoc will be worked out as well, maybe that is usefull later to connect a wireless printer ad-hoc to the core and use cups to send printer data to the printer, and set users that can use hat printer and others can't.

You can ask me on dev channel or here for what you need and i'm glad to get you the data you need.


Title: Re: Link Aggregation in LMCE
Post by: Armor Gnome on August 18, 2015, 01:57:48 am

I like the scope of your plans and projects for more options around network configurations.  Once I've laid out my available time and rebuilt some of my systems to begin testing I'll certainly be in touch with you.

In the mean-time I'll be studying ubuntu network installations as I assume we build after and off of those.  I'll be verifying LAG support packages which I believe should be already present and then looking over lmce usage and needs that your very familiar with in v14.  This gives me some research work before my above proposed implementation strategy.
Title: Re: Link Aggregation in LMCE
Post by: Marie.O on August 18, 2015, 03:40:54 am
We do not use the NetworkManager, but take care of the interfaces file ourselves.
Title: Re: Link Aggregation in LMCE
Post by: Alblasco1702 on August 18, 2015, 04:45:43 pm
Armor Gnome,

we build the /etc/network/interfaces file our self.
the script that builds the file is /usr/pluto/bin/
on this file is the magic to build the network interfaces file.


Title: Re: Link Aggregation in LMCE
Post by: Armor Gnome on August 21, 2015, 12:06:14 am
Being my first project involving true development for LMCE and not just testing or basic feature troubleshooting the research type legwork I am referring to is a better understand of key concepts you all know second hand.  UDEV, modprobe, install order etc are all things I will want to look at before I brush up on bash and dig down to rough in something that not just works but makes sense at every level.

I'm still looking at a few weeks or more until I am ready to begin initial testing and hacking something in to test it manually.  I may be ignorant or unrealistically hopeful but I believe with available information from different sources I can get it working with direct manipulation after installation and my challenge will be in making it sharable as something other than a dirty hack.   Another first for me would be ensuring that any needed packages are available in supported repos and that I understand that process in detail.

You've all been very supportive and helpful so far.  I hope that my desire to understand this process better before I dive in and get stuck shows I appreciate it.