LinuxMCE Forums

So... currently we have to mount the drive momentarily to extract its size. I figured out a trick to get around that, and as always... I need testers. This change will not break anything, and is less invasive than the current method. If it didn't work, it would simply not report the size, or not report it correctly.

That is another thing. I am reporting ACTUAL size... so do not be confused if your 2TB drive shows up as 1.82 TB... for example.

Just backup your current /usr/pluto/bin/StorageDevices_Radar.sh, and replace it with this
http://svn.linuxmce.org/trac.cgi/export/26267/people/l3mce/StorageDevices_Radar.sh

Then either plug in some drives, or remove known drives from your system and let it detect them.

This should work for physical devices you attach to the system (hard drives, usb drives, sd drives, software raids etc), not network devices.
Thank you in advance.

Worked fine for me this morning.  I tested it using a small 8G thumb, and a WD Passport 500G.  On the Passport it detected it as two 250G drives because I have it petitioned for use as backup image storage.

I planned on pulling that box out of the wall for maintenance later this week so will pull a few internal drives and see how it handles missing drives that were detected and mounted prior to your new radar.sh.

Thank you armorgnome!

For the other 75 people who looked at this thread... 1 tester (other than myself) isn't enough. It will only take you a couple of minutes, literally.

Thanks again in advance!

Quote from: l3mce on July 26, 2012, 04:54:48 PM
For the other 75 people who looked at this thread... 1 tester (other than myself) isn't enough.

What happened to rule #1? :p
I intend to test it, but my house is a mess after I have been battling with some hardware issues this week... :)

Updated information on your radar.  It took me a while to realize that this was my culprit but I believe I verified it with a few different restore points and step retracing.

Peek-a-boo internal drives:

Beyond orbiter notification that drives were "re-"discovered and ready to use that has been a mild annoyance for a while and partially resolved with changing the onscreen orbiter notification to "important."  This issue is actually the disconnecting of internal drives routinely and quite often.  While working with files using the media sync area of the admin page, entire drives black out and become unselectable.  Leaving that area and returning finds the drive again or shows another drive black.

While on KDE desktop, all of my sata drives "flicker."  Of my 4 drives, dolphin will show 1, then blink and show 3, then blink and show 2, etc.  This "flickering" or rediscovery of drives happens I would say every 1-2 seconds.  I dont know the frequency of your radar doing its sweep but thought that timing may be important for you somehow. 

General system info:
Hybrid, 4 internal sata drives ( .16T install, .25T storage, .25T storage, 1T storage)
10.04 DVD latest install, frequent updates (typically pulling the latest from sqlCVS and update/upgrade every 2-3 days)
Last clone point where problem did not exist was the 23rd of June.  Looks like I ran your patch on the morning of the 25th.

Believe it or not, the drives mounting/unmounting is the correct behaviour.

This is caused by the fact that we use autofs to automatically mount each disk as needed. This is done with a script (/etc/auto.plutoStorageDevices), which does a database query to determine if a disk is remote or local, and executing the appropriate mount command. When a disk is not used for more than 5 seconds, the disk is umounted.

-Thom

More or less related,
Im keen on having a system that is low on energie usage.
With THE system constantly looking for new media files, disks are constantly spinning.
Is this assumption correct?
If so, is it possible to have the disks only spinning when reading media files or when doing a manual sync of media files.

Br Mathieu

You can uncheck the Update Media Daemon while using Media/Files Sync in the web admin, but you will have to manually sync the files each time you add them. Honestly, I think this is insanely retarded, and I think you're being silly for wanting to keep the disks down for what amounts to a negligable energy decrease (you'd do far better just making sure your light bulbs in the house are migrated to something more energy efficient), but hey, that's just me, what the hell do I know? I mean, I've only been working in this field for 20 years.... ;)

-Thom

To follow up on what Thom is saying,... Z-wave dimmers, in addition to alternative lighting,  can also allow you to save energy with incandescents. There's a negligible difference between an incandescent bulb at 75% of capacity and 100% capacity in terms of light output, but that in of itself yields a 25% savings on the electricity that bulb uses. And 50% provides plenty of light with a 60W bulb, if you're not reading or otherwise need more light. Plus the system can automatically control them... Similarly with thermostats...

Of course, various countries are trying to outlaw incandescent bulbs. It seems they think mercury is better for the environment (not to mention our health) than burning a little more fossil fuel...

Well, this is why I have become a huge champion for LED bulbs...They dim much better, last longer, and no mercury.

-Thom

I was an early adopter, buying 3 or 4 LED lights, and they all burned out on me in less then 3 mos. I don't want to spend the $$$ until I'm positive the dimmable LEDs have improved.

Quote from: tschak909 on July 30, 2012, 06:10:40 PM

. When a disk is not used for more than 5 seconds, the disk is umounted.

-Thom

Thank you for this simple explanation Thom.  I am thinking now of pulling my smaller 250G drives until I fill up my 1T.  This would stop the onscreen notices for them and as the rest of the conversation goes, save a little energy.

Quote from: Armor Gnome on July 30, 2012, 10:48:29 AM
Updated information on your radar.  It took me a while to realize that this was my culprit but I believe I verified it with a few different restore points and step retracing.

Peek-a-boo internal drives:

Beyond orbiter notification that drives were "re-"discovered and ready to use that has been a mild annoyance for a while and partially resolved with changing the onscreen orbiter notification to "important."  This issue is actually the disconnecting of internal drives routinely and quite often.  While working with files using the media sync area of the admin page, entire drives black out and become unselectable.  Leaving that area and returning finds the drive again or shows another drive black.

While on KDE desktop, all of my sata drives "flicker."  Of my 4 drives, dolphin will show 1, then blink and show 3, then blink and show 2, etc.  This "flickering" or rediscovery of drives happens I would say every 1-2 seconds.  I dont know the frequency of your radar doing its sweep but thought that timing may be important for you somehow. 

General system info:
Hybrid, 4 internal sata drives ( .16T install, .25T storage, .25T storage, 1T storage)
10.04 DVD latest install, frequent updates (typically pulling the latest from sqlCVS and update/upgrade every 2-3 days)
Last clone point where problem did not exist was the 23rd of June.  Looks like I ran your patch on the morning of the 25th.

The entire idea behind this new radar was to eliminate ANY mounting/unmounting during the process. This will not be at fault for any such behavior. As Thom explained, this is due to autofs and is the correct behavior. What the radar does, in brief, is make a list of drives, those that are mounted, it ignores. I originally rewrote this because that was not occurring due to an incompatability with the depreciated and dying HAL. We now use udev.
So, finding unmounted drives, they are checked against the database to see if anything has been told to be ignored. If devices, unmounted, not ignored still exist... it should say I found this thing, these are the details of this thing... do you want to use it? Part of that detail is it's size. The only way known by the original folks to cleanly get the size of the device, was to mount it read only and du -h (parsing fdisk is ugly). I discovered a deeper level of udevadm info which exposes this information in an uncomfortable way, and then I convert it a couple of times to make it human readable. This is all I really want to test... that it reports the ACCURATE human readable size. I could have made it report what people want to see... but... I have a penchant for truth in advertizing.

So to sum up... I am curious if this causes problems I cannot foresee, or does not report size on something in some way I did not foresee. I am pretty sure this works as expected.

Quote from: Armor Gnome on July 31, 2012, 12:34:45 AM
Thank you for this simple explanation Thom.  I am thinking now of pulling my smaller 250G drives until I fill up my 1T.  This would stop the onscreen notices for them and as the rest of the conversation goes, save a little energy.

Because they are not mounted does not mean they are not spinning. Yes, pulling them will be the only way to reduce overhead... however they should not continually notify you unless you say "Not now" when asked. If you tell it never to notify you again, it should not... until you unlock it from known devices.

I've had it installed for about a week and have no issues with (dis/re)appearing drives.
Been through router reloads and core restarts and drives I expect to appear do so.

-Coley.

Quote from: Armor Gnome on July 31, 2012, 12:34:45 AM
Thank you for this simple explanation Thom.  I am thinking now of pulling my smaller 250G drives until I fill up my 1T.  This would stop the onscreen notices for them and as the rest of the conversation goes, save a little energy.

If your 250G drives are external, and powered by AC adapter, and you have Z-wave, you could get a Z-wave appliance module, connect a small power strip to that plug those drives AC adapters into that power strip, and you could instruct the drives to turn on and off with the Z-wave system. That way, they'd only be spinning when you need/want them, and you'd only be out the negligible amount of electricity for the Z-wave module. Z-wave appliance modules (the external/internal kind) are only about $45 USD a piece.

To do this transparently, would require some interesting changes to the system to be able to do this, as well as some serious thought. NAS drives could be done this way, because they are autonomous as to their position on the device tree. The device template in question would need to respond to On and Off commands, to power down, and the storage devices scripts would need to be adjusted (augmented) to handle the new device template.

However, I will say this again and again and again, you are barking up the wrong tree here, this approach is retarded, error prone, and ultimately your power savings will be negligable. Stop it.

-Thom

Thom, you are correct... and what I completely skipped over is that you should only shut off the power to the drives after you issued a umount command to them (through drive management) otherwise write-backs wouldn't take and would lead to a disk that would lose data and have to be scanned for errors. Plus,... these external drives typically only consume about 3-5W or less anyway, roughly the same as a typical alarm clock.

However, if he insists, I was just pointing out that there is a way to make the system do what he wants... Although it would be more efficient (& "green") to use Z-wave appliance modules on real resource hogs. ... Things like compressors, amps, fans, etc.

He could also just have the system order MDs to shut down while their users are likely not to be home, and turn back on (boot on network signal) when they're likely to be back home. He can also yank unused HDs from their systems, if they are only going to be network booted MDs.

Quote from: JaseP on July 31, 2012, 07:11:41 PM
Thom, you are correct... and what I completely skipped over is that you should only shut off the power to the drives after you issued a umount command to them (through drive management) otherwise write-backs wouldn't take and would lead to a disk that would lose data and have to be scanned for errors. Plus,... these external drives typically only consume about 3-5W or less anyway, roughly the same as a typical alarm clock.

However, if he insists, I was just pointing out that there is a way to make the system do what he wants... Although it would be more efficient (& "green") to use Z-wave appliance modules on real resource hogs. ... Things like compressors, amps, fans, etc.

He could also just have the system order MDs to shut down while their users are likely not to be home, and turn back on (boot on network signal) when they're likely to be back home. He can also yank unused HDs from their systems, if they are only going to be network booted MDs.

What does this have to do with my script?

If you want to have off-topic "maybe you could" discussions, I insist that you at least test the script and report back before derailing this any further. I do not want to commit this to both 8.10 and 10.04 based on 3 peoples results.

Thanks as always.

Does it need to be replaced only on the core or on the MDs as well???

It should be replaced on all machines that share HDDs drives to the LinuxMCE world.

I switched it on my core and on 1 of my three MDs. Didn't get to the other 2, as there was some family upheaval. For the same reason I didn't have time to test attaching removable storage. They haven't generate any errors, just sitting there though (no news is good news on that front, I guess). I'll have time to update the other MDs and test tonight,... most likely.

Questions:
1) I take it that there's no need to do a reload router or restart seeing as these are shell scripts,... Yes/No??

2) Are there any scenarios that you specifically want to test for,... like attaching, removing and re-attaching storage, removing storage without proper umounting, attaching different partitioning types (ext2/3 versus FAT16/32)??

1) No.
2) Once detected and queried... nothing else matters... other than asking you again, or finding your already existing drives (which it should not).

Ok, I'll report back what I find...

Will, changing the partitioning type (e.g.: go from FAT32 to EXT3 on the same drive and with the same volume name) for a previously detected volume confuse the script an/or database? And/or is that something you would want to test for?

*****************************
EDIT to include findings:
Sorry for the Vebosity in advance

Note: have a net install 10.04, don't know how this will affect results... Also, I only managed to test in Core and the one MD I changed,... To much family drama for the rest of the MDs...

Tested attaching two Compact Flash (yes, those big ol' dinosaurs), a 1GB and a 4GB both formatted in EXT2 (or EXT3?), through a Sandisk USB multi-card reader attached, first, to the Core, and second, to the MD. ...

The Core did not recognize the 1 GB card, neither did the MD (the card had no files on it, only a temp directory, and permissions were root). The 4GB card was recognized as an internal HD by the MD, but not recognized (separately) by the Core (normal?). To be fair, when the system was installed, it tried to recognize the flash card reader as a storage device (no cards inserted, however), and I told it to ignore it. So, the Core may have been responding as expected. I believe I tried the Core first, if that matters.

When the 4GB card was recognized by the MD I selected it to be private to one user (me), and to be used only when I say so. On subsequent eject and re-insert, it did not prompt me for using the drive. Ejecting did not create any message feedback (I do not know if any of this is expected behavior), nor did it remove it from the storage drive list.

Note: The 4GB card, while formatted in a Linux partition scheme, DID have its permissions set liberally for all to see & use. However, it had a copy of Land of the Lost in AVI format on it (if anybody is wondering, I own the DVD, so fair use as far as I'm concerned). The system did not seem to ask anything about the file (Is it even supposed to?).

Again, the cards were formatted in what is typically used for HDs,... I don't know if LinuxMCE makes any assumptions on what a storage device is based on partition type. But again NOTE,... Selecting the drive to be ejected, did not remove it from the HD storage list on screen... I do not know if this is normal behavior.

*****************************

Side question: Is there a way to edit the drive database to make the system see a storage device as a removable device regardless of partitioning scheme? I would kind of like to move that Flash Card to the Compact Flash list, so that the system does not expect it to be attached,... and especially if there are file transfer options that are presented when a Flash Storage device is introduced (& is there such an option?)... What I have done until now is dropped to the KDE desktop on the core if I was transferring any files. I manually rip movies with AcidRip and/or Handbrake, to save drive space & specify media geometry. I don't particularly like DVD menus, extras, alternate language VODs or non skippable  trailers,... and don't want them taking up my HD space. If the system typically responds to removable storage by prompting for file transfers, I am somehow missing out on that feature. It would be nice to know if I'm doing something unexpected, and short circuiting the system functionality in the process.

Note: I updated the above with findings on testing... (long,... trying to be thorough,... sorry).

Updated the script on my core and restarted the machine. My media driver working fine so far except that it still goes offline whenever I restart a media director... hmm. Was sort of hoping the updated radar script might help but maybe I'm barking up the wrong tree.

http://forum.linuxmce.org/index.php?topic=12721