CLOS work on trunk

Over the past few days, work has been going on with respect to our CLOS implementation again. There were a few outstanding issues - some of them rather long already.

One of such longer standing issues was the performance of object instantiation. In the CLHS there's a specific section about the validity of initialization arguments (initargs) and which ones are to be considered valid. A number of improvements have been committed to trunk:
1. The check has been generalized to support all four cases of that specific section of the CLHS.
2. The checks done during object instantiation have been a performance bottleneck ever since they were implemented; as an improvement, a cache has been implemented, speeding up object instantiation significantly: the performance has been doubled (ie the same amount of time now allows creation of twice as many objects).


On a completely different subject (within CLOS), we had an issue where instantiation of an object depending on a forward referenced class caused evaluation to be aborted, without useful feedback. While fixing this issue could have been done quickly and completely unrelated to other changes, I decided to change the behaviour of some of the functions involved according to the AMOP class finalization specification.

So we have taken some small steps on the long (and hard) road toward a performant CLOS/AMOP implementation on the JVM again!

ABCL hash tables: threading and CLOS

More people are starting to use ABCL in threaded environments - I have been doing so myself since spring 2008. My own use revealed some threading issues in the compiler which have been long solved now.

Last April, David Kirkman mailed the list with some threading problems in our CLOS implementation. Because of other work - including implementation of METACLASS support - there wasn't much time to do much about the issue.

This week, he mailed having found the problem - accompanied with a patch with a solution. Because of our need to support the four Common Lisp equality operators, ABCL implements its own hash tables -four of them - with a shared ancestor class which requires implementation of a few primitive operations. Hash access from the Lisp world was being synchronized by the common ancestor. However, in some locations on the Java side the - unsynchronized - primitives were being called directly.

The solution was of course to add synchronization to the primitives as well. Evaluating the result, the new situation was rather unsatisfying: only a single thread could be reading or writing at the same time, meaning only a single CLOS effective method dispatch could be happening at the same time. You probably understand my reluctance to accept the status quo.

The quick solution to the CLOS dispatch problem was to use the java.util.concurrent.ConcurrentHashMap type: we were keying on symbols, which have EQ equility in the Java world (ie in terms of Object.equals()).

That still left the lisp world with rather unsatisfactory threading performance of our own hash tables and since those were repeating large chunks of code in their specialized primitives, I decided to refactor them completely.

The result is - admittedly by looking very closely at ConcurrentHashTable - a single hash table implementation with 4 Comparator classes, which have nearly-lockless reads from all threads. Inserts into the hash table are still synchronized from all threads, but that's expected to be a lesser problem: you'd primarily expect many reads per write in a hash table.

Concluding, the only (known) CLOS threading issue has been fixed - there are presumably many more, please report if you find them - and as a bonus we have a much more efficient hash table implementation.

CLOS performance

Traditionally, ABCL's CLOS implementation has been depicted as one of its weekest spots. The two main reasons being on one hand its speed and on the other the absense of a good MOP.

I'm proud to see that some of the improvements over the past year turn out to be real performance improvers. Ofcourse, there's more to wish for and improvements will keep coming in. However, another improvement of 90% as in the linked post... ?

As shown for a fact, ABCL's CLOS is improving and while possibly still it's weekest spot, it's definitely becoming useable. If you find performance issues, preferably with example code to show the issue, please report to the Armed Bear mailing list (address information on the project front page).

[Note added: Though 90% improvement is hard to achieve, another 30% (24 seconds to 16 seconds) were realised last weekend.]