Prexisting media (Pictures, Movies) on NAS [Solved]

[Trav]

Hi I am currently trying to switch over from the Dark side to LinuxMCE I am well on my way I have my core/hybrid setup with 810 with all the good stuff loaded.  I have a media director which is an Asus EEEbox (works great) my snag is my NAS.  It is a USRobotics NAS Raid system that has a ton of movies and pictures already on it about a 500 gigs shared through windows shares.   The network plug and play works great it recognizes it as a file server and the shares as windows shares.  The problem is that the directories and the files do not show up in the orbiters.  And when I go to the Web Admin the shares are there but when I go to the Files & Media | Media Files Sync it does not recognized the directories and all of the pictures on the root of the shares are not recognized as well.  And when I manually select the type of file as a Picture it still does not show up in my orbiters.  I figure I have missed something  I have searched all over the wiki and the forum for help so hope fully someone here can!!  

[colinjones]

OK, when you told it to use the share it had auto detected, it then prompted you whether to use "the LMCE folder structure" - yes? And you said 'yes'?

If so, then LMCE must have write access to the NAS, and it will create the correct folder structure under that share, and will then expect to find the media in the appropriate folders. If it doesn't have write access it can't create the folders and thus will never find any media (in this mode the intent is that it will ignore all media outside those folders.... and yes, using this mode, rather than the "Public" alternative is far superior)

If it did have write access, and created the folder structure, but you didn't relocate your media into the folders yet, then again, this is why you see no media, and you should start moving the media into the appropriate folders.

Ideally, you should only have one share on the NAS, and it be writable, prior to allowing LMCE to detect it. Allow it to create one set of the folder structure under that share, then relocate your media into those folders, and sit back whilst it is added to the library (this can take a while if you have a lot of media)

But either way, we can fix up whatever state you are in if you can just describe what has been done and created based on what I described above....

[Trav]

OK, when you told it to use the share it had auto detected, it then prompted you whether to use "the LinuxMCE folder structure" - yes? And you said 'yes'?

If so, then LinuxMCE must have write access to the NAS, and it will create the correct folder structure under that share, and will then expect to find the media in the appropriate folders. If it doesn't have write access it can't create the folders and thus will never find any media (in this mode the intent is that it will ignore all media outside those folders.... and yes, using this mode, rather than the "Public" alternative is far superior)

If it did have write access, and created the folder structure, but you didn't relocate your media into the folders yet, then again, this is why you see no media, and you should start moving the media into the appropriate folders.

Well I said no but after reading everything I wish I said yes.  I have been messing with the settings in the web admin so long I don't rember what I have done now.  I would love to rediscover that device if possible.  I have checked the NAS however and it now has the right permissions.  I would like to set it up like you suggested above.

[colinjones]

First you need to delete the file server device (and its child share device below it) from the web admin using the device tree. Do this with your NAS turned off.

Its a bit of a black art, getting re-detection going. You should check the Ignored devices list in the web admin, and remove any entries that relate to your NAS. You may even need to clear your pnpqueue MySQL table, but try that last.

Now do a reload of the router, or better yet reboot. Once all up, turn the NAS back on and cross your fingers! It relies on DHCP mostly for detection, so as soon as the NAS requests an IP address, LMCE should detect it as a "new" device and go through the same process. It might be worthwhile removing any associated DHCP lease from the /etc/dhcp3/dhcpd.conf file as well.

Only tell it to use one share, remove all the other shares from the NAS's management interface, and ensure there is write access to (no username or password) for the share you are picking. Once you have said yes to LMCE's question, it should very quickly create the folder structure. Wait for that to complete.... if it doesn't, there is a problem you need to troubleshoot before proceeding, if it does, then start moving your media into the right folders.

Again, if there is a lot of media, it will take some time before it starts appearing in LMCE.

[Trav]

Ok I took your advice and now I can see media under the video section and the picture section but the files are there 10 or more times.  Also where do I go in the web admin to see the remote shares files and directories for the status or description of each file?  The screen shot below is where i have been going and as you can see the public directory is there but I can not see any thing lower in the directories.  Also sometimes the device 68 which is my windows share is there and sometimes it is not. have any ideals?

[colinjones]

why did you switch to the /mnt folder, click on the link below there that says Switch to /home/public/data, you should never need to go to /mnt. Under the other one you will find your media, navigate using the left pane, then to look at the properties of a file or folder, click on it in the right pane.

Dunno what you've done to get multiple copies, I haven't see that issue for a long time and can't remember any specific causes. But it sounds like you have screwed up your library database in the process of messing around with the shares. Worst case, you may have to reinstall. Or possibly search these forums for the thread on dropping your database, and starting with a fresh one... probably search for terms like mysql and drop

[Trav]

I have reinstalled the device from scratch and rebooted and it has cleaned up the duplicates and I can see my files now on the MD.  I used the Linux file structure like suggested.  However, I still can not see the files or the subdirectories in the web-admin shown below.  I have used the resynchronize button on the individual shares.  Is there a way to see the files in the web-admin to update the tags and remove the files I don't want more quickly then on the MD?

[colinjones]

I can't really explain it. Are you saying that on the MD if you go to pictures, and set the sort mode to Filename, you see picture files and directories that you can navigate and display. But in the same folder on the web admin, there is nothing as per your screenshot?

I would check that the mounts and symlinks are correct - use a terminal to the Linux shell, then cd into the same folder and look around, eg

cd /home/public/data/pictures/Windows share-blahblah
then...
ls -al
inside that directory there should be the same items as you see on the MD in Filename sort mode.

[Trav]

I can't really explain it. Are you saying that on the MD if you go to pictures, and set the sort mode to Filename, you see picture files and directories that you can navigate and display. But in the same folder on the web admin, there is nothing as per your screenshot?

Yeah that is exactly my problem.  I looked around like you suggested everything is there like you expected.  I check the web-admin right after for grins  still same problem.

[colinjones]

look in the device tree for that share, in its device data there is an "Online" tick box, is it ticked? The other thing you can do is run a manual sync on that folder and carefully go through the log file that pops up. Looking particularly for messages not related to specific media files, especially around the beginning and end...

[Trav]

The on-line box is checked. The whole log file is as follows from the resync:

sudo -u root /usr/pluto/bin/UpdateMedia -d \"/home/public/data/pictures/Windows Share-public [80]\"

10   01/04/10 11:20:50.589      Activating Pluto Media Identifier... <0xb6cfb6c0>
10   01/04/10 11:20:50.594      Pluto Media Identifier activated. Extensions 39 <0xb6cfb6c0>
10   01/04/10 11:20:50.595      MediaState::LoadDbInfo ready to run big query <0xb6cfb6c0>
10   01/04/10 11:20:50.654      MediaState::LoadDbInfo got 71 rows <0xb6cfb6c0>
10   01/04/10 11:20:50.655      UpdateMedia::ReadDirectory /home/public/data/pictures/Windows Share-public [80] <0xb6cfb6c0>
10   01/04/10 11:20:50.680      UpdateMedia::ScanFiles dir /home/public/data/pictures/Windows Share-public [80]: files found: 1 <0xb6cfb6c0>
10   01/04/10 11:20:50.706      UpdateMedia::ScanSubfolders dir /home/public/data/pictures/Windows Share-public [80]: subdirs found: 1 <0xb6cfb6c0>
10   01/04/10 11:20:50.706      UpdateMedia::ReadDirectory /home/public/data/pictures/Windows Share-public [80]/317 maple <0xb6cfb6c0>
10   01/04/10 11:20:50.946      UpdateMedia::ScanFiles dir /home/public/data/pictures/Windows Share-public [80]/317 maple: files found: 60 <0xb6cfb6c0>
10   01/04/10 11:20:51.533      UpdateMedia::ScanSubfolders dir /home/public/data/pictures/Windows Share-public [80]/317 maple: subdirs found: 1 <0xb6cfb6c0>
10   01/04/10 11:20:51.534      UpdateMedia::ReadDirectory /home/public/data/pictures/Windows Share-public [80]/317 maple/House damage 9-18 & 9-19  from Isabelle <0xb6cfb6c0>
10   01/04/10 11:20:51.596      UpdateMedia::ScanFiles dir /home/public/data/pictures/Windows Share-public [80]/317 maple/House damage 9-18 & 9-19  from Isabelle: files found: 7 <0xb6cfb6c0>
10   01/04/10 11:20:51.688      Sync'ing db with directory... <0xb6cfb6c0>
10   01/04/10 11:20:51.693      DB sync'd with directory! <0xb6cfb6c0>
10   01/04/10 11:20:51.695      AlarmManager::~AlarmManager start 0x8c6d258 <0xb6cfb6c0>
21   01/04/10 11:20:51.695      lock(0x8c6d29c) (>3) alarm: AlarmManager.cpp:87  <0xb6cfb6c0>
21   01/04/10 11:20:51.695      acquired(0x8c6d29c) #0 (>3) alarm AlarmManager.cpp:87  <0xb6cfb6c0>
21   01/04/10 11:20:51.695      unlock(0x8c6d29c) #0 (>3) alarm: AlarmManager.cpp:87  <0xb6cfb6c0>
21   01/04/10 11:20:51.695      end cond wait AlarmManager.cpp:102 (0x8c6d29c)  <0xb6cfab90>
21   01/04/10 11:20:51.704      unlock(0x8c6d29c) #0 (>2) alarm: AlarmManager.cpp:102  <0xb6cfab90>
21   01/04/10 11:20:51.705      removed from map (0x8c6d29c) #0 (>2) alarm: AlarmManager.cpp:102  was: 2 size, now 1 Rel: Y Got: Y <0xb6cfab90>
21   01/04/10 11:20:51.705      removed from map (0x8c6d29c) #0 (>3) alarm: AlarmManager.cpp:87  was: 1 size, now 0 Rel: Y Got: Y <0xb6cfb6c0>
10   01/04/10 11:20:51.705      AlarmManager::~AlarmManager done 0x8c6d258 <0xb6cfb6c0>

[Trav]

Ok I just did a fresh install using Beta 2 and I still have the same problem can see the files in the orbiter and can use them but they do not exist in the web admin.  To make things worst I did the previous suggested methods on the first try so no messing with anything yet.  I also went to the terminal and checked out the appropriate mapped directories and like you would suspect they are there and everyone has permission to us them on the Hybrid owner of course is root.  Any one have any suggestions?  

[Trav]

OK I have not given up on this problem.  I found a similar problem on another post but his fix did not help me but this quote caught my eye

Lets establish if the standard linkages for media drives are in place. First get the device number of your drive. Now from a terminal session:

ls -al /mnt/device

You should see a list of disk devices in there that are just a number. One number per device, and one of the should be the number you noted above. For instance if the device number is 30, then....

drwxrwxrwx  root root   0    2009-08-15 20:15  30

The first bit are the permissions and ownership - check these are the same as above. (drwxrwxrwx root root) Now...

I did this an I got the following with 35 being my share with pictures that I can see from MD but not web admin

dcerouter_1006050:/mnt/device# ls -l
total 0
drwxrwx--- 5 root 11578 0 2009-12-13 17:09 35
dr-xr-xr-x 2 root root  0 2010-01-08 16:13 37

The I continued to follow the directions bellow:

ls -al /mnt/device/30

You should get a directory listing of what is on that drive starting with public and user_1/2. Do a few more ls's into the directories to confirm that the content is all there. If so, then at least you know that the drive is 1) accessible and 2) mounted correctly.

Now LinuxMCE patches each of the media folders into the correct media library location using symbolic links. Do a,

and I got these permissions and are the same for all subdirectories:

dcerouter_1006050:/mnt/device# ls -l 35
total 0
drwxrwx--- 3 64439 11578 0 2009-11-23 03:15 public
drwxrwx--- 3 64439 11578 0 2009-11-23 03:15 user_1
drwxrwx--- 3 64439 11578 0 2009-12-01 19:11 user_2

Finally when I did this:

ls -al /home/public/data/videos

This should list an entry for each media storage device, and you can see that it is a link back to the /mnt/device location - confirm that all looks fine and is pointing to the correct location. The permissions should look like this:

lrwxrwxrwx root public

Note the first letter is "l" meaning "link". Now try ls'ing inside that link (it behaves just like the directory it points to) and you should be able to see the contents of the drive, but in this new location rather than its mounted location. This will confirm that the link is in place, right permissions, active and pointing at the right thing.

and I got this which looks right:

dcerouter_1006050:/mnt/device# ls -al /home/public/data/pictures
total 8
drwxrwsr-x  2 root public 4096 2010-01-08 16:37 .
drwxrwsr-x 11 root public 4096 2010-01-20 09:18 ..
lrwxrwxrwx  1 root public   12 2010-01-07 17:57 flickr -> /home/flickr
lrwxrwxrwx  1 root public   35 2010-01-08 16:37 Windows Share-public [35] -> /mnt/device/35/public/data/pictures

All that being said it looks like my problem is permissions could that be it? I attempted to change the permissions in the shell using chmod but that did not work and I am afraid to screw with mounting it manually.  Does anyone have any ideals on what I can do?  Can I change the permissions some how?

[Trav]

Ok figured out the problem!  It was permissions.  In the window shares I could not set up the permissions so the mce server could have all the rights it needed (half ass USRobotics interface) so it could read but that was it. I made a NFS share on a seperate machine and was able to do a chmod to give the server what it needed.  I am still using the samba share (i have the option of both one one share on this particular NAS) to access my media through MCE.  Working Great  Wink