This year we had our first ever "gunslinger" event for 3ds Max. This was an opportunity for members of the development team (developers, QA, design, and product manager) to meet a cross-section of 3ds Max users and discuss their ideas, questions, and concerns. We invited several prominent members of the 3ds Max community to meet with the 3ds Max team in the Montreal office for two and a half days. For some it was also an opportunity to meet with the new 3ds Max product manager Eddie Perlberg, though many customers already know him through his work as an application engineer, his blog, or via his classes at Autodesk University.
During the event we discussed the 3ds Max product development process (e.g. how we use the Scrum development methodology and the legal repercussions that prevent us from talking publicly about current or future feature development). Hopefully they better understood why we are sometimes vague about what we are doing, and why it seems to take long (from a cutsomer perspective) to start on new features. We also spent a lot of time learning how they use the product, what their pain points are, and what they want to do more easily in the future.
One question a lot of them had was how we choose what we work on. What surprised me to learn was that many customers didn't realize how influential CER data is for us when fixing bugs and improving stability. A CER (Crash Error Report) is the data sent to 3ds Max when a crash occurs and a dialog pops-up asking if you want to send the data to 3ds Max. We have an official description of the CER workflow on Autodesk.com.
Autodesk receives hundreds of thousands of CER reports annually. This means that we can't follow-up on inidividual CER reports, and instead rely heavily on data-mining techniques. The data goes to a database and is auto-triaged based on the module and auto-linked to related crashes. When patterns emerge, CERs are grouped into defects which are tehen submitted by the QA team to the development team. So as a developer I often see CERs once there are a group in a certain module (e.g. function or DLL) and assigned to a defect, which is usually a long time after it is submitted. At that point I can end up reading hundreds of CER reports one by one trying to find patterns.
Many people realize that 3ds Max 2014 is a very stable release which is due in large part to us working from data provided to us from CER reports. This has helped us identify, prioritize, and fix problems. The most useful thing you can do is provide us with steps that can consistently reproduce the crash. A description of the problem and the worfklow can also help (one of our users keeps us entertained by describing his problems in Haiku). Yes, we do get some data on what you were doing, but the technology is far from perfect. However, even without anything else just clicking "send" with no data still helps us to identify the modules that need to be reviewed more carefully.