Solution: Intermittent/Incomplete Media> Video scan results alpha 2.24

[pigdog]

I've been noticing incomplete, intermittent results when my media/video is searched for movies.

The movies are tagged with artwork (most of them).

At times I might miss 5 to 10 movies listed in the directory.

Other times I might miss 30 to 40 or an entire disk.

Is there a way to force a re-collection?

Manage Drives> Hard Discs> Manage> Refresh doesn't seem to do anything.

On shutdown I did notice CIFS VF: Unexpected lookup error -112.

[colinjones]

Pigdog....

Have you got several different media shares? If a media share is offline during one of the 2min-ly UpdateMedia scans, it will mark all the contained media as unavailable and so it won't appear in the Orbiter. Apparently, when you have 2 or more media shares on your system this can cause the auto mounting system to get screwed up and not mount the shares when needed, artificially causing the above problem. I haven't seen it, but have read comments from Totallymaxed on the subject. Apparently, they use a workaround by setting up a shell to cd into a directory on each of the shares to force it to stay mounted. He did say they were looking at a fix, but I don't think that has gone anywhere yet.

Col.

[pigdog]

Hi Col.
I have done the following on the core/hybrid...

/usr/share/doc/libdvdread3/install-css.sh (this installs libdvdcss2)
apt-get install w32codecs
apt-get install kubuntu-restricted-extras

However, the core/hybrid or MD won't play media from the NAS

The core will not locally play (on installed DVD player) decrypted or encrypted DVD's.

I can locally play decrypted DVD's from MD.

To play encrypted I...

on the MD I added option broadcast-address 192.168.80.255; to the general section of the /etc/dhcp3/dhcpd.conf file.

did apt-get dist-upgrade

did /usr/share/doc/libdevread3/install-css.sh

did apt-get install w32codecs but got per: warning: Setting locale failed.

Encrypted DVD's now play on MD.

For the media share part...

I can let it sit for 5 to 10 minutes, go back in and out, turn off and on NAS, let it sit - reboot Core - whatever.

My best scenario is missing about 5 to 8 movies overall.  But nothing wants to run from the Core or MD from the NAS.

[colinjones]

hmmm pigdog... I'm confused now.... you weren't talking about DVDs encrypted or otherwise in your original post, that was about the media database which is completely separate from the issue you have just posted about.... Don't think I can help you on the DVD question. But re-read my comments on the UpdateMedia/media database/auto mounting stuff, as that may be your solution to the original post...

[pigdog]

Hi Col,

Yes, I have two issues here that I have been struggling with for a while.

To get media re-discovered I've even gone into mySQL pluto_main PnpQueue and removed the IPaddress for my NAS.

I had to reload yesterday because I couldn't get a MD to PXE boot (it didn't think files were there but they were).

I'm just about ready to go back a couple of alpha releases to see if something was introduced in a re-write.

Thanks.

[colinjones]

On the media issue, the pnpqueue is only relevant for pnp discovery of devices, not media. Each time the system detects a device, an entry is added to the queue. If the signature of that device is unknown, then it will go through the process of prompting you to add the new device to your system. If it is known, then it simply marks the device as online.

If you still have that NAS device in your devices tree, then the pnpqueue process will only mark it as online. So first check the device is there, and whether the online tick box is ticked. If it isn't ticked, and ticking it doesn't remain after, say, 5 seconds, then you may have the CIFS/SMBFS issue. You can change the protocol in the share device's device data, it should be set to cifs by default. Change that to smbfs and save, then tick the tick box again. This should then stay ticked.

Once you have confirmed it stays ticked, if you still can't access your media, then I can only assume that it is the automounting process getting stuck, as described above. Reboot, then go to a terminal shell, then cd into a directory on that share:

cd /home/public/data/audio/<nassharename>
then ls -al to confirm you can see the contents. Now just leave that terminal shell open and that will force it to stay mounted. Now continue to use that MD/hybrid as normal, if it is that problem then this should fix it temporarily. If so, then you can set up a script to cd into that folder on reboot as a workaround.

[pigdog]

Hi Col,

O.K.   

These drives are ext3 formatted and the default is CIFS in webadmin view.

I will try what you suggest (tomorrow - got family visiting from Vancouver) and report what happens.

Thank you for your help and patience.

Cheers.

[pigdog]

I booted up the Core and when finished I set up a console in KDE and opened webadmin.

I swapped back to menu and started the NAS.  I let the NAS run for about 5 minutes.

Via Media> Video I checked my listing and I was missing only 5 to 7 movies.

I selected one to play - screen went black after about 30 seconds got can't read disk or dvdcss library not installed.

Menu screen reappeared.

The following was done during initial software installation on the Core/Hybrid and test MD...

/usr/share/doc/libdvdread3/install-css.sh (this installs libdvdcss2)
apt-get install w32codecs
apt-get install kubuntu-restricted-extras

I let the system sit for another five minutes checked Media> Video and same same.

I went into webadmin and checked the device tree. 

The NAS was listed as DLINK-CDBC47 - DLINK-CDBC47 (File Server #1837

The child views where Windows Share - Volume_2 and Windows Share - Volume_1

Volume_1 and Volume_2 are CIFS Filesystem, Readonly - checked and Online - checked.

I did cd /home/public/data/audio/ and cd /home/public/data/videos did a ls in both and no <nassharename> was listed.

I rebooted the NAS, checked again and no <nassharename> listed.

I rebooted the Core/Hybrid.  Booted the NAS.  Waited. Checked webadmin device status.  Checked Media> Videos and still missing 5 to 7.

I did cd /home/public/data/audio/ and cd /home/public/data/videos did a ls in both and no <nassharename> was listed.

I rebooted the Core/Hybrid.  Booted the NAS.  Waited. Checked webadmin device status.

I did cd /home/public/data/audio/ and cd /home/public/data/videos did a ls in both and no <nassharename> was listed.

I booted a MD.  Tried to play movie via Media> Video.  Screen went black. After about 1 minute movie started, froze, skipped, froze, etc repeatedly.

[pigdog]

Hi,

Found it.

I was looking around in webadmin> Files & Media> Media Files Sync.  I guess since it's a NAS it's in /home/public/data/other.

I have 2 listings.  Windows Share-Volume_1 [34] and Windows Share-Volume_2 [33].

From one terminal window I cd into the directory for Volume 1.

Tried a ls, tried ls -al but got cannot open directory .: No such file or directory.

I rebooted the Core/Hybrid and restarted the NAS.

When Core/Hybrid came up I started the NAS.  I opened a terminal in KDE desktop.

Got into Volume 1 and did ls -al.  This time go listing. 

Opened another terminal for Volume 2. Got listing.

Had to do a quick Regen All.  Got back to main menu.

Went back and did ls -al on each terminal window. 

On Core did Media> Video.  Some movies still missing even though I saw them in directory tree in terminal view.

Selected a movie - screen went black.  After 5 minutes still black.  Did F1> Off.  10 -15 seconds later main menu back.  (Aeon Flux)

Booted up a MD. Selected a movie. (300) It started and played fine.  I watch about 1 minute did F1> Off.

Went to Core tried movie 300.  Ran but freezing, skipping.  Did F1>Off.

Went back to MD.  Chose 300.  Did not run and eventually got Main Menu with can't read disk or dvdcss library not installed.

Went back to Core/Hybrid tried 300 again.  Waited.  Waited.  Nothing.  Did F1>Off.

Aeon Flux is volume 2.  300 is on volume 1.

Device tree shows...

DLINK-CDBC47 - DLINK-CDBC47
   - Windows Share - Volume_2
   - Windows Share - Volume_1

[colinjones]

OK, if its only missing some media but not others off the same share, then it isn't your devices or the CIFS/SMBFS thing.. didn't realise that was the situation.

But if it is in Other, then this means you didn't add the shares as "use LMCE's folder structure" you must have used "make all media public" and that will be your problem for certain files missing.

If at all possible, you should _always_ add your shares using the LMCE folder structure. The other issue is they are read only. You need to make them r/w, and then add them using LMCE's folder structure. This way, LMCE will be able to create the correct folder structure to find your media. Then just move the files into the appropriate folders. And let UpdateMedia discover them.

Not using the LMCE folder structure means that all media MUST be properly tagged to show up in the Orbiter. Those files that aren't appearing are probably not tagged properly. Do the above and you will be able to see them in the Filename sort mode irrespective of tagging.

But this doesn't explain why you are having issues playing them. Firstly, don't test with DVD rips, to eliminate one variable. Only test using .avi's or something. I still think there is the possibility that the issue Totallymaxed has described is causing this other issue. And certainly, the not using LMCE folder structure is the reason you cannot see the shares under /home/public/data/video ... go to LMCE folder structure, and the symlinks will appear in those folders, then you can CD into them to hold the shares mounted permanently.... then test again with avi's ... that should fix that issue.

Only once you have done these 2 things should you move on to troubleshooting the DVD issue so that you can be sure nothing else is causing your issues...

[pigdog]

O.K.  Thanks.

I'll do that.  I find it odd though that this is the first time I have experienced this adventure.

I've always marked the media Public and never used the LMCE folder structure structure (710 and previous 810 alphas).

I was worried that if I used LMCE folder structure my data might get corrupted.

All these files are VOB's about 1.5TB in total.

If I change the file structure and make all media public will certain files still go missing (if properly tagged)?

The MD played ripped and encrypted DVD's locally but when it screwed up playing NAS media I got the ccsdvd message.

Well, let's see what happens.

Thanks for your help.

Cheers.

[colinjones]

Seriously, do not use "make media public" unless you have a _real_ reason to do so. I don't see one here (they are rare, and realistically only if you are using another media source such as NAS where another application _insists_ on a particular folder structure that is different from LMCE. Otherwise, use LMCE's structure)

LMCE does not change your media files at all with the exception that meta tag changes in mp3 files (only) will get updated on the mp3 file itself... so no, it won't corrupt anything.

If I change the file structure and make all media public will certain files still go missing (if properly tagged)?

these are opposites, you choose LMCE's folder structure as an alternative to making all media public (don't get confused with other locations where LMCE talks about making things public, such as ripping disks, this is purely when you add a new share. So you need to delete these shares from your system, and allow it to redetect, then choose LMCE's structure, not public. Make sure the shares are r/w first!)

I don't know how VOBs are handled, but really, if you are ripping disks you are far better to rip them using LMCE's ripper as .dvd files (same as .iso files) as you are eliminating a potential complexity... there is no reason to have dozens of VOB and other DVD files hanging around, they are opaque and confusing. Far better to have a single .dvd/.iso file that is treated exactly the same as the VOBs but simpler and less likely to get screwed up. I realise that you have already got 1.5TB of these, but this is the best advice I can give you.

[pigdog]

Hi,

I deleted my NAS from the directory tree.  Had to go into mysql pluto_main and delete the device from in there to get rediscovered.

When the NAS was rediscovered and I selected the LMCE directory file structure I was not asked to make data Public.

Under /home/public/data/videos (audio, documents, etc) I have my Windows Share-Volumes_1 [65] and 2 [64].

FYI  - ls -al shows total 0 and

drwxrwxrwx 2 501 501 0 date time .
drwxrwxrwx 8 501 501 0 date time ..

On the NAS (viewing under win2k) each volume has a new directory called

user_1
   -data
        -audio
        -documents
        -games
        -pictures
        -pvr
        -videos

All my VOB's are still there.  Just can't see them via LMCE.

Maybe there is some way I can convert them?  Then I won't have to delete them off the disk and reinstall the original hard copy back onto the disk converted as a .dvd file.

Something to maybe try.  Not much to lose I guess except time and effort.

Thanks again.

[colinjones]

The user_# folders are for private media for your individual users (which doesn't work all that well)

There will also be a public folder on your NAS at the same level as the user_1 and user_2 assuming that you have read-write access to the share so that LMCE can create it. If not, make sure you grant read-write access to the shares, then manually create a "public" folder at the same level, then create the same folder structure as you see under user_1. Now relocate all your media from user_1/data/audio|video|etc into public/data/audio|video|etc - remember, if you move the files (not copy) it will effectively be instant and not require more disk space on the NAS.

Once this is complete, on your core under /home/public/data/audio|video|etc/Windows Share-Volume_1 [65]/ you will see your media.... this is how LMCE works, the remote shares are mounted onto each MD including the core/hybrid under /mnt/device##, it then symlinks each audio|video|etc folder into the same /home/public/data path so that all your shares, wherever they are physically located around the house, all appear to be in the same location under the /home/public path.... making your media library consistent, and so that every MD can access the media files using the exact same path.... its clever like that!

Now, once they are there, your UpdateMedia daemon will eventually scan them into the database. This will take a while. But only once this is done will you see them in the Orbiter media grids. (actually you will see each folder appear as it completes that folder). But remember, you must change the Orbiter's Sort menu (bottom left in UI2 and in Options in UI1) to Filename. Now you will see all your media.

Make sure it all appears first, only once you have confirmed that this issue is resolved, move on to the unreliable play issue.....

Unreliable play - try the CD (change directory) method I detailed earlier into the /home/public/data////etc folders cd into one folder per share only.

Retranscoding.... that is going to be a lot of work, as transcoding takes a long time. If you really want to, you can probably use a combination of PowerISO and Handbrake. PowerISO should allow you to present a set of VOBs as a virtual DVD drive, and then use Handbrake to transcode them into a single AVI. But you'll have to experiement as I haven't used them for a little while.

[pigdog]

Great info.  I'll look at this tomorrow.

I just found this link ...http://wiki.linuxmce.org/index.php/Convert_Audio_TS_/_Video_TS_folders_to_.dvd_iso_file

which once everything else is done I could try out.

Once again, thanks for all your help and guidance.  It is appreciated.

Cheers.