Topic: ****WARNING**** SERIOUS GLD80 CRASH | Allen & Heath Digital Community

WARNING SERIOUS GLD80 CRASH

This topic has 17 replies, 12 voices, and was last updated 11 years, 1 month ago by johnfrusketti.

Viewing 15 posts - 1 through 15 (of 18 total)

1 2 →

Author

Posts
2013/04/20 at 5:38 am #23725

mabomer123
Participant

I’m submitting this warning to all GLD80 users of software V1.10 and most likely V1.11. We’ve experienced 2 GLD80 crashes in the last week. Basically, you power down the console using the normal shutdown procedure and the next time you power cycle it goes into a continuous boot process. It begins some count down with
BUG: Scheduling While Atomic error,message continuously scrolling across the screen. I was also informed another local church who bought their board a month before us also experienced this crash. Our board crashed Tuesday night right before rehearsal for a big charity event on Friday. Fortunately, we were able to get a loner console which I spent several hours creating shows on Wednesday night. Yes I did have backups but when I tried to load them in all but 1 displayed an error message and would not load the show. Fortunately, all was not lost, a teen show I had created did load so at least I didn’t have to start from scratch. So finally, tonight was the show and everything seemed to boot up normally. I loaded the headliners show and was giving instruction to their soundman on how to use the GLD80. The band finished their sound check and everything appeared normal but when I went to save their show, it would not save to the hard drive. Also, my other show I created for the opening band that I run sound for was gone. I then tried to load my backups from the USB stick but there was an error message each time I tried to copy or open it. I was in a serious panic and to make things worse I had to listen to the headliners soundman, who is an older gent, drone on and on and on and on about how bad and unreliable digital boards are. The good news is I was able to quickly modify the headliners show for the opening band.
Monitors and the Aviom mix was a challenge but I was able to make it through both shows. After all the trouble, the headliners soundman ask me to mix their show, which I was happy to do since I was so embarrassed about what had happened. I did not reboot the board out of fear it might not come back. So at the end of the night, I shut the board down and sure enough it will not boot up again. I don’t think there is anything you can do to prevent this from happening and I haven’t been able to find anything on the A&H website about it, a work around or anything helpful. It’s a role of the dice. I suggest if possible, don’t shut your board down until there is a fix. If this happens to you, your screwed and if it happens before a show GOD help you. Allen and Heath had better put out a patch or work around fast or there is going to be some serious damage to the confidence people have in this console. Don’t get me wrong, up until now I absolutely loved it. I realize there is always going to be bugs when your dealing with software but hey, at least when my Presonus board crashed I could just wipe it out and reload the firmware. As long as I had my shows backed up I was good. I sincerely hope, that no one else has to go though what I’ve been through this last week.

Best regards,

Martin

2013/04/20 at 6:49 am #33764

Chris93
Participant

Have you contacted A&H, or your local distributor?

Chris

2013/04/20 at 10:36 am #33765

Volum
Participant

Hi Martin.

Sorry to hear about your problems, but it all sounds so familiar I am afraid…
You can read my post regarding similar problems here:

https://iliveforum.allen-heath.com/topic.asp?TOPIC_ID=2465

It was shipped to service after the first time, came back, worked (just tried it at home) a couple of days, put it away for a week, and when i should go to work it failed again.
I got a brand new mixer from the dealer, and it has worked for two gigs now, so I hope it will continue this way…

The dealer wanted to change some of the hardware in my old mixer (they got it back and wanted to use it as a demo-mixer after repairing it). It seems to me that your problems also may not just be software-related, but I’m no expert though…

Anyway: Best of luck to you. Keep us posted on how things turn out []

PS: I also come from the Presonus (SL24) and had to hire one for the two gigs that my old GLD let me down, and you know what: it worked every time, also the Capture-software, and VSL control were hassle free. DS

Willy
FOH-tech.
GLD-80
AR2412
AR84

2013/04/20 at 1:31 pm #33766

Moparcar
Participant

I too have experienced several crashes of our GLD. The original issue was deemed a hardware defect and the desk was replaced a few weeks ago. Now, the new desk has crashed on me. First it kinda lurched then just stopped everything on the surface for a few minutes then resumed on it’s own. The second was mid second set and it never recovered, however the mix was decent at that point and I let it fly until the end of the show as opposed to stopping between songs to reboot. I like the desk, but the reliability issue is becoming fairly serious. Am hoping for another firmware update to deal with these problems soon. Making my company look more closely at the Midas Pro2 regardless of additional expense.

“Can I get some talent in the monitor, please.”

2013/04/21 at 3:37 am #33769

mabomer123
Participant

Update: I got in touch with our locale A&H support reps., who by the way provided awesome support and they were able to get me up and running again. It turns out there is an issue with the USB drives and the hard drive filling up with garbage. I don’t know all the specifics but once the files are removed the board boots normally. I’m sure A&H will explain it better in their release notes in the upcoming maintenance release. Anyway the support was awesome and our local support team were working on the problem by early morning. The good news is, there is a work around so if you run into this problem get into touch with A&H ASAP and they will get you back up running in no time.
The other thing they told me was, don’t remove a USB drive before it completes the read/write cycle. I’ve never done this but apparently this can cause a similar failure. So I thought it would be good to mention on the forum for other GLD users.

Martin Bomer

2013/04/21 at 8:27 pm #33776

Chris93
Participant

Glad to hear it got sorted.

Could the workaround be posted here for reference or is it dependent on the specific crash? If there’s a way to fix it I’d like to be able to fix it on the spot.

Chris

2013/04/22 at 12:09 am #33782

papromike
Participant

This issue is caused by some of the questionable USB keys out there, they create a repeating temp file in the subdirectory that uses up the memory on the desk.

THere is a manual “erase temp file” and reboot proceedure, next week I have been told that there is a V1.2 maintenance release coming out which will automatically clean up these temp files and resolve the issue, sorry youve had some problems, but if you need to get the system up, please contact tech support and they can send you a procedure to boot it up, when you can install the new firmware and youll be good to go.

Vice President of Sales
Allen & Heath USA

2013/04/22 at 2:13 am #33784

Chris93
Participant

Did I read here that Kingston and Sandisk are the way to go? I have an 8GB DT101.

Chris

2013/04/22 at 10:35 am #33789

bucks
Participant

Hi All

Some more information on Martin’s crash.

There are 2 separate bugs here which are unrelated but cause the same point of failure:

1) An O/S indexing service generating large temporary files
2) USB Key hot unplug generating excessive event log entries

Both eventually cause the on board storage to fill up and the desk will fail to boot.

Martin’s failure was type 1, but type 2 is more common.

V1.12 maintenance release contains 2 changes to address these issues:

1) We’ve disabled the indexing service, to prevent it creating temporary files.

2) We check the log file size on the next boot, and clear them up if they are preventing the system from booting correctly.

The thing to note here is the log files ( for type 2 ) are created because the system was busy writing to a USB key, and the key was removed before the write was completed.

Our modification in V1.12 will therefore get a console broken in this way working again AFTER a reboot, but doesn’t prevent it from happening in the first place. Further after the disk has filled up, and before the console has been rebooted, the console will be unable to store shows / libraries to the disk – its full!

To prevent this situation you need to wait for the write to USB Key to completely finish. There’s usually a light which flashes on the device, but often the write activity happens in bursts with gaps in between. This can cause the user to think its finished, when actually its just gearing up for another write!

The USB Key manufacturer has a slight bearing because some are faster than others. You are less likely to hot unplug a fast key, because its more likely to be finished! Also some manufactures use large caches to improve write performance, and haven’t actually completed the write, but tell the console they have!

In addition to the fixes in V1.12 we’re looking into suppressing the event log entries altogether for a future firmware update which would stop the event log from overflowing.

But …

The user will still run the risk of loosing data if the key was removed before the write completes.

The best advice is therefore to give the USB key writes a bit of breathing space. Count to 10 before pulling the key out after a write!

V1.12 should go live on the website later today.

Hope this helps.

Andy
A&H

2013/04/22 at 11:34 am #33790

jdh28
Participant

quote:

Originally posted by bucks

Hi All

Some more information on Martin’s crash.

There are 2 separate bugs here which are unrelated but cause the same point of failure:

1) An O/S indexing service generating large temporary files
2) USB Key hot unplug generating excessive event log entries

Both eventually cause the on board storage to fill up and the desk will fail to boot.

Andy,

Thank you for the detailed response and for being transparent about the issue.

regards,
John

2013/04/22 at 2:01 pm #33791

RYAN-LOUDMUSIC-JENKINS
Participant

Andy, was this caused by bugs in V1.11 or is this in all versions?

Thanks, Ryan

2 X GLD80
2 X AR2412
1 X AR840

2013/04/22 at 2:55 pm #33792

bucks
Participant

Hi Ryan

All versions.

Although for the 1st year in the field these problems were unheard of.

It may just be the number of units in use has statically started to highlight these issues, or there may be localized factors which make the manifestation more prevalent.

In any case upgrading to the latest is recommended.

Cheers

Andy.

Andy
A&H

2013/08/04 at 3:37 pm #34544

xj32
Participant

This just happened to me this morning and I am on Firmware 1.20.

My local distributor/sales rep has no idea how to solve it at this point. It would have been nice if the fix – reboot proceeder would have been posted here, as we are a mid size church, its Sunday morning and we are dead in the water for the day.

Paul

2013/08/05 at 4:04 pm #34562

bucks
Participant

Hi Paul

I’m sorry to hear you’re having problems.

The code changes discussed above were introduced in V1.12 so you already have them.

The procedure for the log file cleanup is simply to power cycle the desk.

If this doesn’t resolve the issue, there is something else at play.

Either way it would be best to contact Allen & Heath Tech Support and open a ticket so your issue can be investigated.

Hope this helps.

Andy
A&H

2013/08/05 at 11:00 pm #34568

John-S
Participant

The crash has been pretty much documented and solved as much as any software can.

After A&H releases a new firmware update they are pretty much absolved of issues due to old firmware. It is incumbent upon the owners to keep the firmware up to date. If issues happen on new firmware then that is a different story.

Isn’t the heading of this post unnecessarily dramatic and alarmist?

John Stefanick
https://www.RentWirelessMicrophones.com
Author

Posts