ftocc

"Too many files in. . ."

katse
Posts: 37
Member Since:
2006-10-16

Every so often I find myself having to clear files in [..]monitor/ and [...]voicemail due to the message ' To many files in /var/spool/asterisk/monitor/'

So my question(s) is(are) this:

What is introducing the limitation on the number of files? Certainly not the filesystem. . .

Where can this limit be raised, and how?

If this is set artificially low to begin with and the system can actually track more, why is it set so low to begin with?

Any help is appreciated, I am turning friends onto asterisk (the Trixbox distro) as their office solutions and to have them see this pop up when I show our system is a bit of egg on my face. :D

By the way: aside from echoing issues with the TDM4 cards, great job guys. Can't beat trixbox at any price!



katse
Posts: 37
Member Since:
2006-10-16
Re: "Too many files in. . ."

No one else has encountered this issue?



UtahLuge
Posts: 3
Member Since:
2006-10-31
Re: "Too many files in. . ."

I am getting this SAME EXACT issue. We are about to try just rotating out the files but there is probably some setting in a configuration file. Please if you know please please please help us! Thank you!!!!



katse
Posts: 37
Member Since:
2006-10-16
Re: "Too many files in. . ."

No one else is encountering this?



katse
Posts: 37
Member Since:
2006-10-16
Re: "Too many files in. . ."

Will Trixbox 2.0 fix this?



UtahLuge
Posts: 3
Member Since:
2006-10-31
Re: "Too many files in. . ."

I don't know about Trixbox 2.0, but I have found some stuff where there is a variable in the config file. I added slashes to it instead of dashes but no luck. And its the ONLY place I have been able to find something. Maybe you know of a place where the variables are?
This is basically what I have found:
----------------------------------------
/etc/asterisk/extensions_additional.conf

exten => 500,n,Set(MONITOR_FILENAME=/var/spool/asterisk/monitor/q${EXTEN}/${TIMESTAMP}/${UNIQUEID})
----------------------------------------

----------------------------------------------------------------------
/usr/src/freepbx/amp_conf/htdocs/admin/modules/queuesfunctions.inc.php
$ext->add('ext-queues', $exten, '', new ext_setvar('MONITOR_FILENAME','/var/spool/asterisk/monitor/q${EXTEN}/${TIMESTAMP}/${UNIQUEID}'));

$addarray = array('ext-queues',$account,'3','SetVar','MONITOR_FILENAME=/var/spool/asterisk/monitor/q${EXTEN}/${TIMESTAMP}/${UNIQUEID}','','0');
----------------------------------------------------------------------

----------------------------------------------------
/usr/src/freepbx/amp_conf/htdocs/admin/functions.php
$addarray = array('ext-queues',$account,'3','SetVar','MONITOR_FILENAME=/var/spool/asterisk/monitor/q${EXTEN}/${TIMESTAMP}/${UNIQUEID}','','0');
----------------------------------------------------



UtahLuge
Posts: 3
Member Since:
2006-10-31
Re: "Too many files in. . ."

Oh ya, what file system are you using?
I am using Ext3. One of our developers told me something about the Ext3 filesystem only allowing X many files in a directory. He said something about that being the reason for the "To many files" error. That is also the reason I tried to get each extension, and date, into its own directory.



katse
Posts: 37
Member Since:
2006-10-16
Re: "Too many files in. . ."

I'm using ext3. If this is the cause for the limitation, is there any way to force Trixbox to install on ReiserFS or XFS?



katse
Posts: 37
Member Since:
2006-10-16
Re: "Too many files in. . ."

Anyone?

anyway, some more info: I had to clear the monitor directory again, I had to delete everything prior to September.

Here's the thing though: it is still recording perfectly fine, so I don't think it's a filesystem error. I think it's a FreePBX error. I am not familiar enough with the project to know where to look for the code, but is it possible there is a limitation in any arrays used to enumerate the files prompting this error? It's frustrating because when we have to revisit calls to verify orders to make sure we get things right we're unable to review the calls until I get a chance to go in and purge older files. Since it's still happily creating the voice recordings it's obviously not a filesystem issue.



katse
Posts: 37
Member Since:
2006-10-16
Re: "Too many files in. . ."

OK. when I posted I forgot that php does not care about types. Now I am thinking memory, so I will check memory utilization and php.ini settings next time I run into this (should hit the limit again within a week).

Any thoughts on the matter?



katse
Posts: 37
Member Since:
2006-10-16
Re: "Too many files in. . ."

Running into this again. I edited php.ini:

memory_limit = 128M

And restarted httpd. Same issue, still encountering the error.

I find it hard to believe that no one else is encountering this issue. We're hitting this after a couple of months of calls with only three lines seeing regular use.

OK, after about two minutes of digging we found it

edit: bootstrap.php

In: /var/www/html/recordings/includes

It is hard-coded to not handle more than 3000 files.

Line 112,

        if ($fileCount>3000) {
          $_SESSION['ari_error']
            .= _("To many files in $msg_path Not all files processed") . "<br>";
          return;
        }

We're changing this to make it a setting. The question is: why is this limitation here in the first place?



nose13
Posts: 45
Member Since:
2006-06-15
Re: "Too many files in. . ."

maybe the better question is why are there so many files in there?

I've checked all of our trix boxes and we don't have any files sitting in there. Some are heavily used 30 station boxes, some are tiny little 2 stations boxes.



katse
Posts: 37
Member Since:
2006-10-16
Re: "Too many files in. . ."

Do you do call monitoring? These are all valid WAV files. If you have nothing in /var/spool/asterisk/monitor then you do not have call monitoring enabled. We have it enabled because we have to test it for support for our clients, and for "quality assurance purposes" (e.g., when we bring in reps we need to make sure they're not trying to screw or piss off the customer, etc.), typical call center stuff. Our clients actively use it, we have it enabled for testing but will be actively using it starting in the coming months.

So the question is not why are there so many files in there (that's kind of a 'duh' question, to be quite honest, and I truly mean no offense there) but why is that limit encoded into the php in the first place? It is obviously not a filesystem limitation because ext3 can handle many tens of thousands of files in a directory, and it's obviously not a kernel limitation in this case (the ONE article I've found on this issue claimed it was a kernel limitation, requiring a reconfiguration/recompile). Did someone, when implementing ARI, think "gee, 640K, er, 3000 files ought to be enough for anyone?" ;)



agit8or
Posts: 261
Member Since:
2006-05-31
Re: "Too many files in. . ."

Katse:

We gave up on using ARI on any call center or high call volume installation. It simply can not keep up due to the fact it reads the directory when the script is executed. IMO, the best way to track recorded calls would be to create a link (with the filename) in the CDR and have that link clickable once the cdr was viewed. We changed the cdr display page to read recorded calls live (like ARI) right from the directory, but it too was SLOW once the files accumulated.



TheShniz
Posts: 213
Member Since:
2006-06-01
Has anyone found a solution to this as yet?

Has anyone found a solution to this as yet? I just ran into it for the first time with one of our call centers running call logging (aka monitoring). Did raising the artificial 3000 limit relieve the problem?

- J



TheShniz
Posts: 213
Member Since:
2006-06-01
double post

double post



TheShniz
Posts: 213
Member Since:
2006-06-01
bumpity, anyone? - J

bumpity, anyone? - J



imagex
Posts: 108
Member Since:
2006-06-06
Any Fix for this?

Any Fix for this?



Yondaime
Posts: 24
Member Since:
2007-05-31
The ext3 filesystem has a

The ext3 filesystem has a limit on how many file entries are in a directory. I think the PHP limit is there to warn you if the number of recorded conversations is already approaching that limit.

Try using cron to move the recording files to a different directory each day. That should solve the directory/file number limitation.

--

Erol M. Fornoles



Yondaime
Posts: 24
Member Since:
2007-05-31
Here is a small script that

Here is a small script that I schedule via cron which segregates the recording files to different directories based on the date. It's still not robust, and I appreciate any contributions to it.

md_monitor.sh
---
#!/bin/bash

ls /var/spool/asterisk/monitor | grep 'OUT' | awk '{print "[ -a /var/spool/asterisk/monitor/"substr($1, 9, 8)" ] || mkdir /var/spool/asterisk/monitor/"substr($1, 9, 8)""}' | sh
ls /var/spool/asterisk/monitor | grep 'OUT' | awk '{print "mv /var/spool/asterisk/monitor/"$1" /var/spool/asterisk/monitor/"substr($1, 9, 8)"/."}' | sh

--

Erol M. Fornoles



percykwong
Posts: 753
Member Since:
2007-04-30
I raised my limit from 3000

I raised my limit from 3000 to 30000 and all is well.

--

-----------------------------------------------
Percy Kwong
www.swimminginthought.com
www.iphonebounties.com



Zion800
Posts: 69
Member Since:
2007-03-06
I was having the same

I was having the same problem and made the same edit and I stopped getting the error.



mekashron
Posts: 7
Member Since:
2007-04-24
Limition

I have found that if you rise the limit to a large number a timeout error is displayed.

Yondaime: Can you adjust the script to move old wave files into backup directory like every new month create new backup directory.

Also where can I register this file to auto run every X days?



Zion800
Posts: 69
Member Since:
2007-03-06
I think I just realized why

I think I just realized why we get the error "Too many files in....".

I looked in /var/spool/asterisk/monitor, and for some reason, there is another 'monitor' directory in there, with the same exact recordings (same inodes). So, when the ARI goes to read the directory, it keeps reading through all of the subsequent "monitor" directories that are nested within each other. I'm no Linux expert, so I don't want to mess with the /var/spool/asterisk/monitor/monitor without consulting with the forum first.

What do you think? Is there a way we can delete it without breaking everything?! :p



Zion800
Posts: 69
Member Since:
2007-03-06
Just thought I'd let people

Just thought I'd let people know, the monitor directory I mentioned in my previous post is a symlink. I removed it and increased my limit from 3000 to 300,000,000 and everything is working flawlessly (as far as I can see).



Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.