Concerning the Development of AXE 4

AXE 4 has been under development (mainly by me) for quite some time and to be honest it will continue under development for quite some time to come. There is an awful lot of code in AXE and even a straight port would be difficult; a complete rewrite in C#, with a very different approach to optimization and scalability than the original AXE, is a huge task. I'm in no hurry.

Why .NET?

I'm really not interested in discussing the merits of .NET in general. So, talking about AXE specifically...

AXE 3 is written in MFC. MFC is dead. MFC is also very very difficult. At the time AXE was made, MFC was the best performing and most flexible GUI toolkit in the world, but life is simply too short to plunge into the hell that is MFC again.

Java does not work for applications like AXE. Java does not deal well with raw memory, which is what AXE mostly deals with. Java does not deal well with low-level type casting, boxing and unboxing, which are all critical areas for AXE. Java is hard and slow to interoperate with OS specific code, which is something AXE has to do a lot of, if features like editing process virtual memory are to be kept. Also, there's nothing Java offers that .NET doesn't.

The main weakness of C# is performance. It's faster than Java but much slower than C#, although this slowness isn't always concentrated where you would expect. Currently, after a lot of work and investigation, I believe that C# is fast enough (on modern machines) that very intense operations such as finding periodicity of data, and mapping variable length structures onto data can be done in C#.

Aren't there any unsolved problems with .NET?

Yes, there are. In particular, disk access seems to be slower in .NET and tasks like 'grep the entire hard drive for the string "bananas"' are much much MUCH slower in AXE 4 at the moment than in AXE 3.

Another problem is with the GUI -- not only do you need to trust the user to have a .NET runtime, you also need to pick between the various .NET GUI kit offerings, which are not very mature. Currently, it looks like AXE4 will be doing a lot of manual drawing to the screen, just like AXE 3.

Why a complete rewrite?

Times have moved on since AXE 3 was designed. Some parts of AXE 3 are very robust and scalabale, others are not. And other hex editors have caught up, too. AXE 4 needs to be a whole level more flexible and modular than AXE3.

A key point to make here is that I am not trying to make AXE 4 some sort of Computer Science nightmare, so flexible and scalable and generic that it never actually does anything. I see enough projects like that, thank you very much. AXE 4 won't be beautiful, it will be as functional as it needs to be.

It still needs a rewrite, though. Look at the grammar generator in AXE 3. Good, isn't it -- except that it lives in its own special window and barely communicates with the rest of the program at all, and the infrastructure just isn't there to make it do so. That level of infrastructure does have to be introduced.

What will be different?

Ooh, a great deal.

Structure System

The structure system will be greatly refactored, such that structures and basic types (a basic type being something like '4-byte unsigned integer') are much more similar to work with. The user should be able to search for structures, for instance, and easily declare nested structures or structures whose composition is decided dynamically based on the data.

File Geography

Some data has 'geography'. For example, PE files (Windows code files) are divided into segments and a great many other things. .NET assembly files have even more internal structure, and so do document files, image files, 3D models, etc. In AXE 3 the best you could do to express this was putting bookmarks all over the file. AXE 4 will have a much deeper system, in which a file can be annotated with regions, structures, and marks as it is loaded.

Large Data Sets

There really will be no limit on file size this time around.

Masks

AXE 3 has 'masks' such as the Diff Mask, the Changed Mask (which makes text appear in red) and the regular plain old Mask (which is shown with a grey background). These masks are not very efficient and there's not that much you can do with them. Even so, the Mask is used a lot to map out an area of the file based on manual selection and searching. In AXE 4, there will be a much more efficient and flexible mask system, so that it will be easy to mark a given region of the file, then further mark bytes in that region that match a particular search... then further mark those that meet some other criterion... and so on.

How can I help?

How nice of you to ask. I desperately need advice, thoughts, and comments. Not code; most of the code that can be fully specified has been implemented but there is so much left to specify. Email me or leave a comment, please, if you have any suggestions on:

• Dealing with the filsystem; historically AXE has always left this to WinHex but maybe it's time to start?
• GUI framework -- what framework to use? DotNetMagic seems good, but what is its long term potential?
• Workflow: in AXE4, you can search for a partly-completed structure. Sounds good -- but what would the UI look like for that? It would have to let you pick a structure, and then for each member, express what range of values are accepted, and then -- it would be very very complicated, is about all I know.
• Algorithms -- there's some good stuff in AXE 3, but AXE 4 should really be intelligent enough to divide a file into regions based on variations in periodicity, entropy etc.