I've realized that sometimes a message can block the queue, to the point where the system cannot process any more messages. Incoming messages get stuck in the queue (behind the offending one, I assume), while outgoing messages get stuck "somewhere" in "Pending" state.
I am forced to restart the application to fix this, at which point:
a) messages to be sent are marked as "Failed"
b) messages in the processing queue are (apparently) lost
Needless to say, this puts a damper in any unattended system, or forces me to monitor the log files for this anomaly.
1) Any ideas at what may be happening?
2) Any ideas at how to peek into the incoming processing queue? Any test code, log, etc will be most welcome.
Here is what I have in my trace log, I'm trying to find the "root" message, but it has been overwritten already (+420MB of logs, no sermons please):
[Incoming message processor] TRACE [11/01/12 17:06:45] net.frontlinesms.messaging.IncomingMessageProcessor.run(IncomingMessageProcessor.java:123) - Getting incoming message from queue.
[Incoming message processor] TRACE [11/01/12 17:06:45] net.frontlinesms.messaging.IncomingMessageProcessor.processIncomingMessageDetails(IncomingMessageProcessor.java:165) - Got message from queue: 21447570:1950753
[Incoming message processor] DEBUG [11/01/12 17:06:45] net.frontlinesms.messaging.IncomingMessageProcessor.processIncomingMessageDetails(IncomingMessageProcessor.java:170) - Sender [+251913339673]
[Incoming message processor] WARN [11/01/12 17:06:45] net.frontlinesms.messaging.IncomingMessageProcessor.run(IncomingMessageProcessor.java:145) - Error processing message. It will be queued for re-processing.
org.springframework.dao.IncorrectResultSizeDataAccessException: Incorrect result size: expected 1, actual 2
Thanks for getting in touch about this - not sure what the problem would be here, but will pass this one on to our developers to see if they can help get to the bottom of this.
I wonder if this is related to the issue I'm having.
Sounds quite familiar, I leave my system unattended which then leads to messages being stuck on "Pending".
It looks like this is a bug in the handling of delivery reports. We'll look into getting it fixed, but for now you may find that disabling these reports will stop the issue occurring. You can do this via the settings for your phone/modem.