Magical dissappearing media!

[lmwashere]

Strange,

Every few days, most of my media seems to drop out of the database. This seems to mostly affect video files, pictures and games, but apparently not the music files as they have yet to disappear.

I can browse right to the missing media from the kde desktop and it is there if I check it through the admin website or just over the network. If I re-sync the media through the website, it all shows up fine again. At least for a few days.

OR if I leave it alone, then several hours later, most or all of it will all usually come back.

This seems to have started after I elected to install mame from the admin website. At least it did not have an episode before then, and the machine had been running for two weeks or so on this install prior to that.

Shortly after the mame install it told me that it had detected a new internal HDD and asked me what I wanted to do with it. Now I had not installed a new HDD, but due to me having to reinstall pretty regularly with my experimenting and all, I have my media saved to another partition on the same physical disk. In any case I told LMCE that I wanted to use it and to make all of the media public. Then all of this craziness started.

In the past I have had issues when using an external HDD of the drive constantly mounting and dismounting and I am wondering if perhaps this may be the source of the issue. Maybe it dismounts it and the media drops out, then remounts it and I happen to look for a movie prior to it putting it back in the db. Is that even plausible? Seems like a stretch. Especially since the music files are on the same partition and seem to always be present. Should I mount the partition to the home folder during the install, or would that run the risk of corrupting the system as would an external drive?

Josh

[totallymaxed]

Strange,

Every few days, most of my media seems to drop out of the database. This seems to mostly affect video files, pictures and games, but apparently not the music files as they have yet to disappear.

I can browse right to the missing media from the kde desktop and it is there if I check it through the admin website or just over the network. If I re-sync the media through the website, it all shows up fine again. At least for a few days.

OR if I leave it alone, then several hours later, most or all of it will all usually come back.

This seems to have started after I elected to install mame from the admin website. At least it did not have an episode before then, and the machine had been running for two weeks or so on this install prior to that.

Shortly after the mame install it told me that it had detected a new internal HDD and asked me what I wanted to do with it. Now I had not installed a new HDD, but due to me having to reinstall pretty regularly with my experimenting and all, I have my media saved to another partition on the same physical disk. In any case I told LinuxMCE that I wanted to use it and to make all of the media public. Then all of this craziness started.

In the past I have had issues when using an external HDD of the drive constantly mounting and dismounting and I am wondering if perhaps this may be the source of the issue. Maybe it dismounts it and the media drops out, then remounts it and I happen to look for a movie prior to it putting it back in the db. Is that even plausible? Seems like a stretch. Especially since the music files are on the same partition and seem to always be present. Should I mount the partition to the home folder during the install, or would that run the risk of corrupting the system as would an external drive?

Josh

Hi Josh,

We've seen this before too and from memory it was a problem in the Storage_Radar script... it is continually un-mounting & re-mounting your recently added partition. I thought a fix for this was already in place to be frank. I'll check on Monday and post back here.

All the best

Andrew

[tschak909]

It has to do with the automounter. There is a timeout of a few seconds of inactivity, before the volume is unmounted. This is to support devices that need to be unplugged etc, such as network devices etc.

We need to modify UpdateMedia to be more aware of this, and to prevent a race condition where it thinks the files are not there, when in reality they're just not mounted due to the timeout..

We need some help identifying and fixing the problem, any help from anyone with experience here or just a willingness to dig into the problem would be much appreciated.

-Thom

[colinjones]

thom - wouldn't the UpdateMedia process itself cause autofs to remount the drive, and therefore be visible as UM scans? The impression I got was that when autofs umounts a drive, the drive still appears to be there in the path, but just not actually mounted, and it only physically disappears from the path when you actually disconnect the drive. So wouldn't UM think it is still there, attempt to scan it, and that action of 'touching' the drive path remounts it in time for UM?

So is UM thinking the drive is there and mounted, as the path exists, but manages to scan the path and see there are no files before autofs remounts the drive (rather than UM thinking the drive is actually fully unmounted)? Or is something else happening here I haven't understood?

[tschak909]

that is a very likely scenario. we need help to properly diagnose and solve the problem.

-Thom

[colinjones]

obviously I'm not the one for that, but perhaps a kludge, interim solution would be to force mount all devices immediately prior to UM running, ugly but would probably work for now.

Looking at the autofs config it looks like the timeout is 30 seconds. Looking at the auto.master config and our autofs script in /etc/auto.PlutoStorageDevices reveals a pretty hefty script which includes SQL queries, etc to identify the correct LMCE devices to mount.... I'm assuming that running this script is causing the race condition you refer to, its non-deterministic whether the script completes and the mount finishes before UM actually looks at the path and sees there are no files.... I guess that means that UM does indeed think the device/drive is mounted but that the files have gone missing (rather than the device simply not being available) and that would cause a significant amount of SQL activity, marking all those files as missing, rather than just the device being off line....

[tschak909]

yup, this will be an interesting issue because autofs removes the file entry completely...

-Thom

[Xtra]

On a sidenote, what amount of timeout are we talking about here?

I ask because I see mount and unmount requests on my NFS server every few seconds.
It goes like:

Code: [Select]

Dec 20 15:31:05 valhal mountd[3708]: authenticated mount request from 192.168.10.112:718 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:31:12 valhal mountd[3708]: authenticated mount request from 192.168.10.112:923 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:31:15 valhal mountd[3708]: authenticated mount request from 192.168.10.112:919 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:31:22 valhal mountd[3708]: authenticated mount request from 192.168.10.112:891 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:31:25 valhal mountd[3708]: authenticated mount request from 192.168.10.112:761 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:31:33 valhal mountd[3708]: authenticated mount request from 192.168.10.112:834 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:31:35 valhal mountd[3708]: authenticated mount request from 192.168.10.112:713 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:31:44 valhal mountd[3708]: authenticated mount request from 192.168.10.112:1017 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:31:46 valhal mountd[3708]: authenticated mount request from 192.168.10.112:734 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:31:54 valhal mountd[3708]: authenticated mount request from 192.168.10.112:769 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:31:57 valhal mountd[3708]: authenticated mount request from 192.168.10.112:942 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:32:04 valhal mountd[3708]: authenticated mount request from 192.168.10.112:791 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:32:06 valhal mountd[3708]: authenticated mount request from 192.168.10.112:974 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:32:09 valhal mountd[3708]: authenticated unmount request from 192.168.10.112:869 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:32:14 valhal mountd[3708]: authenticated mount request from 192.168.10.112:756 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:32:16 valhal mountd[3708]: authenticated mount request from 192.168.10.112:897 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:32:25 valhal mountd[3708]: authenticated mount request from 192.168.10.112:801 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:32:28 valhal mountd[3708]: authenticated mount request from 192.168.10.112:668 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:32:34 valhal mountd[3708]: authenticated mount request from 192.168.10.112:831 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:32:38 valhal mountd[3708]: authenticated mount request from 192.168.10.112:797 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:32:46 valhal mountd[3708]: authenticated mount request from 192.168.10.112:716 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:32:50 valhal mountd[3708]: authenticated mount request from 192.168.10.112:893 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:32:59 valhal mountd[3708]: authenticated mount request from 192.168.10.112:850 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:33:00 valhal mountd[3708]: authenticated mount request from 192.168.10.112:720 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:33:02 valhal mountd[3708]: authenticated mount request from 192.168.10.112:750 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:33:10 valhal mountd[3708]: authenticated mount request from 192.168.10.112:1006 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:33:12 valhal mountd[3708]: authenticated mount request from 192.168.10.112:784 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:33:20 valhal mountd[3708]: authenticated mount request from 192.168.10.112:930 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:33:22 valhal mountd[3708]: authenticated mount request from 192.168.10.112:822 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:33:30 valhal mountd[3708]: authenticated mount request from 192.168.10.112:742 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:33:32 valhal mountd[3708]: authenticated mount request from 192.168.10.112:668 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:33:40 valhal mountd[3708]: authenticated mount request from 192.168.10.112:777 for /mnt/data1/movies (/mnt/data1/movies)
Dec 20 15:33:42 valhal mountd[3708]: authenticated mount request from 192.168.10.112:973 for /mnt/data1/movies (/mnt/data1/movies)
As you see, there is a radical imbalance between mount and unmount.

Any ideas how to pinpoint the problem?

[totallymaxed]

obviously I'm not the one for that, but perhaps a kludge, interim solution would be to force mount all devices immediately prior to UM running, ugly but would probably work for now.

Looking at the autofs config it looks like the timeout is 30 seconds. Looking at the auto.master config and our autofs script in /etc/auto.PlutoStorageDevices reveals a pretty hefty script which includes SQL queries, etc to identify the correct LinuxMCE devices to mount.... I'm assuming that running this script is causing the race condition you refer to, its non-deterministic whether the script completes and the mount finishes before UM actually looks at the path and sees there are no files.... I guess that means that UM does indeed think the device/drive is mounted but that the files have gone missing (rather than the device simply not being available) and that would cause a significant amount of SQL activity, marking all those files as missing, rather than just the device being off line....

A simple temp fix would also be to create a screen session that accesses /home/public/data so that the file system is never unmounted

Andrew

[lmwashere]

Well over time the disappearing media issue seems to have self resolved, or at least mutated. Now I am getting multiple copies of movies showing up in the videos section. A few minutes ago I had 25 listings for "The Iron Giant".

If I open the movie from the LMCE menu it lists a circular location like /public/data/other/(name of hard drive (device 64)/public/data/other/(name of hard drive (device 64)/public/data/other/(name of hard drive (device 64)/public/data/other/(name of hard drive (device 64)/public/data/other/(name of hard drive (device 64)/public/data/videos/movie name

The original movie is stored in /public/data/videos

If I go into /public/data/other and delete the shortcut back to the hard drive, then rescan media, all the extras go away and it behaves properly again. The problem is that I have to do this over and over again because LMCE apparently likes to put those shortcuts in there. Is there any way that I can keep it from putting those shortcuts in the /other folder?

Could I change the owner and permissions of the /other folder without shooting myself in the foot later. Also is this where it is supposed to add external hard drives?  I do not want to screw up the usage of my 500 GB HDD, (not the one that these movies are stored on) by trying to fix this recursive link problem.


Thanks in advance,
Josh