ColdFusion Muse

Finding all those sneaky log files in Coldfusion

At some point you will have to troubleshoot server problems. Trust me - it will happen. It's no good sticking your head in the sand or your fingers in your ears (or both - tricky, but doable). Your first question will likely be, "Where do I find out information about what is happening on the server." Fortunately there are a great many log files in Coldfusion. Unfortunately they are not easy to find, and they contain different formats of information. There are 3 "groups" of file based logs on a Coldfusion MX installation. These groups are mostly the same for CFMX 7 as they are for CFMX 6 and CFMX 6.1 (a few exceptions). Here's a rundown:

1. /CFusionMX/logs - Coldfusion Logs

Actually, most CF Folks already know about this location. It contains the following logs:

  • Application Log - This is a record of nearly all the errors that are thrown by the server. I say nearly because errors thrown by onSessionEnd() in the application.cfc file are not recorded here for some unknown reason (see this entry). Errors generated by Flash using Flash remoting are not recorded here either. The errors are in a single line comma delimited text qualified format. They look like this:
    "Error","jrpp-17","06/15/05","14:14:56","ap_v1","Variable undefined... "
    "Error","jrpp-17","06/15/05","14:15:48",,"File not found: c:\..."
    "Error","jrpp-14","06/15/05","14:16:37","ap_v1","Error Executing..."
    "Error","jrpp-22","06/15/05","14:19:41","ap_v1","Element undefined...
    
    The jrpp-N identifier is the processid or thread number. The date and time followed by the name of the application (if germane) and the error message. The file path is also included, which is why you should take pains to protect this file from prying eyes.
  • Exception Log - This log is a more complete version of the application log. It includes Flash remoting errors on a CFMX 6 server (but still no "onsessionEnd()" errors on a cfmx 7 server). This file includes the stack trace of the error in question. Using the date, time and the jrpp (thread or process id) you can get more specific detail regarding errors in the application log. I use this log when the error thrown is too generic to be of use. Sometimes the Java stack trace will provide additional clues. this log resembles the following:
    "Error","jrpp-2134","06/08/05","10:38:00",,"File not found..."
    coldfusion.runtime.TemplateNotFoundException: File not found: /members.cfm
    at coldfusion.filter.PathFilter.invoke(PathFilter.java:83)
    at coldfusion.filter.ExceptionFilter.invoke(ExceptionFilter.java:50)
    at coldfusion.filter.BrowserDebugFilter.invoke(BrowserDebugFilter.java:52)
    ....
    
  • Server Log - This log contains the start up routine detailing the start of all the services related to CF. It's really only useful when you are troubleshooting specific services (like the graph server for example). The one exception is that this is the location that CF logs slow pages. If you set the parameter to log pages that are slower than N number of seconds you will see warning messages here that say "blah.cfm" took longer than N seconds.
  • Mail Log - This log contains errors that are reported from sending mail. You will see things like "invalid address" posted here. You can adjust the level of mail logging in the CF Administrator.
  • MailSent Log - If you set CF to log all sent mail you will see the messages logged here. The messages are logged with the TO, FROM and SUBJECT.
  • Flash Log - This log (as far as I can tell) contains errors generated by server side flash compiling, like the chart engine or flash forms (as of CFMX 7). in CFMX it also contains errors generated by RMA (flash remoting), but those are in a separate log in CFMX 7. If your information is different or more specific, please correct me on this one.
  • Flash-remoting (cfmx 7) - This log looks very much like the exception log except it contains errors that are generated by flash remoting.
  • Eventgateway Log (cfmx 7) - If you have the enterprise version of CMFX 7 you will see this log. It contains errors and exceptions fired by the event gateway. If you are on standard version you will see a message stating that your license is invalid - it means the license for the gateway.
  • Additional Gateway Logs (cfmx 7) - If you are running gateways you will see a log for each one listed here. You may also see a gateway specific exception log.

2. /CFusionMX/runtime/logs - Jrun Logs

This only applies if JRUN is your underlying Java Ap server (as in the standard edition or Enterprise w/ JRUN). In this folder you will see at least 3 logs. They are named for the name of the "server" or "servers" located in the /runtime/servers/ folder. The naming convention is "*servername*-err.log (or -out.log or -event.log etc). The server name is actually the name of a folder inside of the "servers" folder that contains configuration files for the server in a SERVER-INF subfolder. This is a byproduct of convoluted Java folder management, and one of the frustrating things about Java (where to find everything). So, for example, if there were 2 servers in the /runtime/servers/ folder called "server1" and "server2" you could expect to see "server1-out.log" and "server2-out.log" in the /runtime/logs folder.

2 hints for those of you running standard version CF. CFMX 6.x uses a server titled "default" so you will see "default-err.log", while CFMX 7 uses a server titled (more appropriately) "coldfusion" so you will see "coldfusion-err.log". The information placed in these logs comes from the JRUN server. It may or may not reflect errors in the coldfusion log, but it is usually lower level information, useful in troubleshooting the JVM and server level performance issues

  • *servename*-err.log - You will find socket connection errors, compiler errors and other non-specific CF errors in this log. If you had a server crash, it's a good place to look for symptoms. The format is not text qualified.
  • *servername*-event.log - You can actually edit the jrun.xml file to "split out" the various events into separate files. These events may or may not be error events. This log contains startup and initialize events, Warnings and other server level info.
  • *servername*-out.log - This is the Java "standard output" log. Basically, anything that needs a home but doesn't tie to a specific log goes here. That makes the information useful, but the format is unpredictable. There are CF errors logged here as well - perhaps because the "out" logger isn't aware of the CF logging operations. I'm not sure.
  • *servername*-metrics.log - To see this log you must specifically enable metrics (see this excellent how-to by Brandon Purcell). There is a whole group of metrics you can log. Much of the information is similar to what you might find using the performance monitor or CF Stat - things like threads, idle time and memory usage. It's a good way to monitor if you are experience outages at regular intervals.

3. /CFusionMX/db/slserver52/logging

This one is more obscure and not often looked at. This is logging by the sequelink ODBC server. If you are using the ODBC socket driver or any Access DB's on your CF installation you might find useful information here. On a CFMX 7 (or CFMX 6.1 with ver. 5.4 drivers) the path is /CFusionMX7/db/slserver54/logging. You will see 2 types of files here - log files and "dmp" files (dumps). The log files have errors generated by the sequelink server and the dmp files are created when there is an exception resulting in a crash. I have yet to find the .dmp files useful. Usually there is stack trace and an access violation - not much more.

Obviously, in addition to these there are loggers you can set up and customize on your own. There are also the OS logs (event viewer for example) and the web server logs. After knowing where all the information is located the real issue becomes sifting through it. On that note I'll give this piece of advice. If you are troubleshooting a crash, go to each log and extract the information that ranges from 20 minutes before the crash to the recovery (reboot or whatever). Don't draw any conclusions or concoct any theories until you know everything that was going on at that point in time on the server. Then, create scenarios that you think are possible causes and try to figure out a way to test them or (worst case) trap them the next time they occur.

One of the biggest errors I see in troubleshooting is folks who start with 1 piece of information and as soon as they have a guess they go with it. This produces shallow solutions that often do not take all the variables into account. When you are dealing with a crash or performance sometimes having patience is the hardest thing to manage - but it can be your greatest asset.

Related Blog Entries

Comments
Steven Erat's Gravatar Ooooh goodie! This is like a log file easter egg hunt. I like that game.

Seriously, working in Tech Support I can tell you I eat, sleep, and dream ColdFusion log files. Inside out, back and forth.

I like this post, and I think I'll use it to recommend to customers. Many times when we provide explanations or solutions that reference Macromedia docs its percieved with a bit of skepticism, so an "outside" or "third party" resource like this helps overcome that.

Oh, and its Brandon, not Bruce, Purcell ;)

Note that the CF logs and the JRun logs exist in both Server configuration installations and multiserver. In server config the JRun logs for ColdFusion MX 7 are coldfusion-XXXX.log, while in multiserver its cfusion-XXXX.log for the "default" CF instance.

CF on Solaris and Linux has a cfserver.log file in the server configuration and multiserver config. Its very similar to the "servername"-event.log, and I like to use it get basic information about port usage, build numbers, interal service startups, etc...

In support we almost never use the Sequel Link logs (db/slserver). We really push JDBC hard, and advise against desktop databases like FileMaker Pro, Foxpro, or MS Access.

Don't forget install logs too. The basic installation alone will lay down a Macromedia_XXXXXXXX.log in the server root directory, and running the migration wizard will create a migration.log and maybe migration exception log in the main CF logs dir. If you install with debug options, you get an additional install_debug.log.

I used to like the CF5 admin.log (or cfadmin.log?). It would log every setting on a CFAdmin settings page when any setting was touched. It was a great record to see how settings got changed, and its often reveal some "oops, I didn't know that got changed" moments. That is missing from MX... hope it gets brought back.

Fusebox apps were always especially hard to troubleshoot since the errors always regarded index.cfm, given that the query string controlled the view.

ColdFusion 5 and earlier used to log SQL statements, template names/paths, and generally much more than CFMX. The internal feedback I got was that logging output changes aren't considered bugs per se.

One of the most useful techniques that I've found is to review the main server settings (Settings Summary Report) and then look at all the logs while focussing on a specific time window. Its often possible to build a picture of the events that occurred by reconstructing the errors from the various logs into one body so they can be viewed in sequence without having to dig through alll the log content and all the log files. Just extract snippets and rearrange them chronologically in one file.

Not sure if I added anything here, but great post!
# Posted By Steven Erat | 6/23/05 10:49 AM
Mkruger's Gravatar Awesome - thanks "brandon". Sorry about getting your name wrong. I corrected it. Thanks for the additional information you added - very helpful indeed.
# Posted By Mkruger | 6/23/05 11:41 AM
Steven Erat's Gravatar I'm Steve, actually, not Brandon. I was just pointing that out :)
# Posted By Steven Erat | 6/23/05 12:19 PM
Greg's Gravatar I found a link to some software that sounds like it is designed to find CF problems. Something called FusionReactor (fusion-reactor.com) which does stack traces and memory checking and stuff like that. You guys seem like you are experts on the subject, could you maybe give your opinion on how useful it is?
# Posted By Greg | 12/1/05 5:15 PM
Dan Gleneck's Gravatar How about adding some simple coldfusion code / web page that displays these logs using cfdirectory and cffile ?
# Posted By Dan Gleneck | 9/2/09 6:22 PM



Blog provided and hosted by CF Webtools. Blog Sofware by Ray Camden.