the Lemacs/FSFmacs schism

The Lemacs/FSFmacs Schism.
© 2000 Jamie Zawinski <jwz@jwz.org>

In 1991, I was working at Lucid Inc., and our newest product, Energize, was an integrated development environment for C and C++ on Unix. The design of this development environment involved very tight integration between the various tools: compilers, linkers, debuggers, graphers, and editors. So of course we needed a powerful editor to tie the whole thing together, and it was obvious to all of us that there was only one editor that would do: Emacs.

At the time, the current version of GNU Emacs from the FSF was Emacs 18. There was another version of GNU Emacs called Epoch, that had been developed by Alan M. Carroll (and later by NCSA). Epoch was a set of patches to Emacs 18 that gave it much better GUI support (Emacs 18 was very much a tty program, with GUI support crudely grafted on as an afterthought.)

For the last few years, Emacs 19 had been due to be released ``real soon now,'' and was expected to integrate the various features of Epoch in a cleaner way. The Epoch maintainers themselves saw Epoch as an interim measure, awaiting the release of Emacs 19.

So, at Lucid we didn't want to tie ourselves to Emacs 18 or on Epoch, because those code bases were considered obsolete by their maintainers. We wanted to use Emacs 19 with our product: the idea was that our product would operate with the off-the-shelf version of Emacs 19, which most people would already have pre-installed on their system anyway. That way, Energize would make use, to some extent, of tools you already had and were already using.

The only problem was, Emacs 19 wasn't done yet. So, we decided we could help solve that problem, by providing money and resources to get Emacs 19 finished.

Even though Energize was a proprietary, commercial product, all of our work on Emacs (and on GCC and GDB) was released under the GPL. We even assigned the copyright on all of our work back to the FSF, because we had no proprietary interest in Emacs per se: it was just a tool that we wanted to use, and we wanted it to work well, and that was best achieved by making our modifications to it be as freely available as possible. (This was one of the earliest, if not the earliest, example of a commercial product being built to a significant extent out of open source software.)

Well, our attempts to help the FSF complete their Emacs 19 project were pretty much a disaster, and we reached the point where we just couldn't wait any longer: we needed to ship our product to customers, and our product needed to have an editor in it. So we bundled up our work on GNU Emacs 19, called it Lucid Emacs, and released it to the world.

This incident has become famous as one of the most significant ``forks'' in a free software code base.

When Lucid went out of business in 1994, and I came to Netscape, I passed the torch for the maintenance of Lucid Emacs to Chuck Thompson (at NCSA) and Ben Wing (at Sun), who renamed it from ``Lucid Emacs'' to ``XEmacs.''

To this day, XEmacs is as popular as FSFmacs, because it still provides features and a design that many people find superior to the FSF's version.

I attribute Lucid Emacs's success to two things, primarily:

• First, that my focus was on user interface, and an attempt to both make Emacs be a good citizen of modern GUI desktops, and to make it as easy for new users to pick up Emacs as any other GUI editor;
• Second, that I ran the Lucid Emacs project in a much more open, inclusive way than RMS ran his project. I was not just willing, but eager, to delegate significant and critical pieces of the project to other hackers once they had shown that they knew what they were doing. RMS was basically never willing to do this with anybody.

Other things that helped Lucid Emacs's success, but were probably less important than the above:

• We gave the users what they wanted first. People had been anticipating Emacs 19 for years, and we stopped dragging our feet and finished it. So this got us a lot of users up front. However, XEmacs's current popularity can't be attributed to this, not since 1993, anyway.
• Lucid Emacs was technically superior in many ways. This won us the mindshare of many good developers, who preferred working with Lucid Emacs to FSF Emacs. It would be nice if technical superiority was all that mattered, but realistically, the other factors were probably more important than this one, as far as number of users is concerned.

The following messages, from the Lucid Emacs mailing lists in 1992 and 1993, comprise the bulk (if not the entirety) of the public discussions between the Lucid and FSF camps on why the split happened and why a merger never did.

The current XEmacs maintainers have a much more pusillanimous summary of this history on their XEmacs versus GNU Emacs page.

-- jwz, 11-Feb-2000.

Dramatis Personae:

Richard Stallman: creator of Emacs, founder of the FSF.

Jamie Zawinski: that's me, I did most of the hacking on Lucid Emacs and managed the project. Near the end of my tenure, dozens of people were working on Lucid Emacs on a regular basis, and I reviewed and tested all the code they emailed me, as well as stewarding all the mailing lists. (This was in the days before public CVS servers...)

Richard Gabriel: founder of Lucid Inc., my boss, and long-time friend of Richard Stallman.

Richard Mlynarik: prolific contributor to Lucid Emacs: he did almost all of the work of merging changes that were made to the FSF version back into the Lucid version. He was the largest contributor to Lucid Emacs who wasn't actually getting paid to work on it.

Marc Andreessen: you've probably heard of this guy, but you probably didn't know that he used to be one of the maintainers of Epoch at NCSA.

Joseph Arceneaux: the fellow initially employed by both the FSF and by Lucid to hack on Emacs 19, before I started working on it.

Jim Blandy: employed by FSF to hack on their version of Emacs 19, after Arceneaux.

Kyle Jones: author of VM, one of the Emacs mail readers.

Dave Gillespie: author of ``calc,'' a scientific calculator application that runs inside of emacs; calc was hit harder by Lucid Emacs compatibility problems than most other Emacs packages.

The Emacs Timeline also might be helpful.

It is on labrea.Stanford.EDU (36.8.0.47) in pub/gnu/lucid/.

What's new:

• I've rearranged the lisp directory so that there are only a handful of files at top-level; everything has been moved to subdirectories. This should speed up the automatic generation of load-path, since it will have to stat() 10x fewer files.
• You can now input and display all iso8859-1 characters. If you set the variable ctl-arrow to something that is non-nil and non-t then high-bit characters will be displayed in the current font instead of in octal.
• If you load x-compose.el, you can do Sun/DEC-like compose processing. See the comment in that file.
• Random bug fixes, installation problem fixes, some portability #ifdefs.
• The pre-built sun4 binaries are compiled with optimization now.

What's old:

• It doesn't use "configure" yet.
• The OLIT menubar still doesn't work. Next release for sure...

Enjoy...

-- Jamie

I am curious as to what people think of the likelyhood of EMACS and Epoch merging into a single release.

Isn't that what's called Emacs 19?

Further, it would be nice if any new Lucid-emacs features get rolled in. It's getting confusing with Emacs 18.X, Epoch, Emacs 19, and Lucid Emacs based on 19 all floating around at the same time. For instance, I make frequent use of ^Z2 to dup a screen in Epoch..it's something else in lucid emacs, and I haven't used Emacs 19 [1] but I'm wondering if it's yet another thing.

[1] I gather 19 really isn't widely available yet.

Anyways, I think competition is a good thing and all, but I'm also hoping it doesn't start getting out of hand. Are we going to have three different versions of Emacs under the GNU license? From my perspective as a relatively new user of Emacs (about 1 year and 9 years of Vi before that), I can keep the differences straight in my mind, and there's ways of binding keys so that everything looks the same, etc. However, if I'm explaining something about Emacs versions to someone who's new to all of this, I can see contortions on their face, etc :-) It's starting to look like the history of some rock bands (like what's happened over the years to the band "Yes").

Would it be worthwhile for people to work towards a common GNU Emacs goal?

What does Richard Stallman think of all of this?

I'm just tossing this out there. If there's going to be 2 or three GNUish Emacii, fine, I can keep that straight. I just personally hope there isn't any more splintering. Perhaps there should be an Emacs Summit Meeting? Anyways, this is all from a user's perspective who is wondering what is going on, but who doesn't necessarily know all of the history behind it...so keep any flames informative rather than outright hostile :-)

Daniel

Would it be worthwhile for people to work towards a common GNU Emacs goal?

I don't think so. If anything I think it would be worthwhile to consider pulling Emacs apart into its components, much as OS developers have pulled apart operating systems. There are some good ideas embodied in Emacs. I think they could be improved by distilling them out of the monobloc (to horribly mix metaphors). But this isn't the place for such discussions.

From: dansmith@autodesk.com (Daniel Smith)
Further, it would be nice if any new Lucid-emacs features get rolled in. It's getting confusing with Emacs 18.X, Epoch, Emacs 19, and Lucid Emacs based on 19 all floating around at the same time. For instance, I make frequent use of ^Z2 to dup a screen in Epoch..it's something else in lucid emacs, and I haven't used Emacs 19 [1] but I'm wondering if it's yet another thing.

How many incompatibilities are really present, yet?

Emacs 18.58 and Epoch 4.0 are identical, except for the Epoch extensions.

Emacs 19 isn't out yet.

So current incompatibilities include exactly these:

• Differences between Epoch and Lucid X commands and related interfaces.
• Differences in general between Emacs 18.58 and Lucid Emacs 19.

The Lucid folks are, I think, doing a good job of handling the latter problem, while the former problem could be temporarily solved with an Epoch emulator package for Lucid (an idea which has been tossed around on help-lucid-emacs) or vice versa.

[1] I gather 19 really isn't widely available yet.

Isn't available at all yet, basically.

Marc

--
Marc Andreessen
Epoch Development Team
University of Illinois at Urbana-Champaign
marca@ncsa.uiuc.edu

The long delay in releasing Emacs 19 is the FSF's fault. (In some sense, therefore, mine.) While it's regrettable that there are multiple versions, I can't blame people for filling the gap that the FSF left. One of the goals of the copyleft is to allow people to do this--so that one central maintainer's lapse does not hold back the rest of the community.

However, the people who wrote the Lucid version didn't try to to make the changes usable for us. They charged ahead without discussing the design with me or with the main Emacs maintainer. They reimplemented things we had already done. I asked them to do some work making parts usable for us, but they said no.

The result is that not much of the code in Lucid's version will ever appear in the FSF's version. Some of the features are worth having, but need to be reimplemented. If you'd like this to happen sooner, one way is to volunteer to do some of the work.

Emacs 19 already has most of the features of Epoch.

A couple of rocks from a glass house[*]:

a) It would be _really_ useful if someone familiar with emacs19, leamcs and epoch could give everyone an idea of what the major differences are. Not so much details, but places where the major abstractions differ. For instace lemacs `faces' and epoch `styles' look fairly similar, converting would be irritating but not suicide-inducing. I havn't looked at screen creation etc in lemacs in detail but from what I have seen it treats screens as parallel to windows while epoch treats them as parallel to buffers (current-screen vs selected-screen etc), if that difference goes deep it might be very painful.

b) What would really be good would be emacs19 simulators for both epoch and lemacs. I'm less interested in getting at existing epoch code than in giving my own code a future path.

[*] There _is_ going to be a combined up to date tvtwm soon, honest.

- RJC

I read RMS's message with sadness. Over the past two years I have tried very hard to figure out a way to work with FSF on Emacs, and for a lot of that time I thought we were working together, but apparently I failed. For those of you who wish to read about that effort, you can read the postscript to this note, which outlines Lucid's involvement - a sort-of history of the events.

We still want to work with FSF and have our contributions become part of FSF Emacs, and we are eager to hear suggestions on how to proceed.

-rpg-

P.S. RMS's letter paints an unflattering and unfair picture of Lucid's involvement with Emacs, and in particular on the motives and efforts of our hackers.

The Free Software Foundation is a remarkable organization - for many years I have supported it personally (and still do), and my company supports it, its goals, and its products. This support is in terms of time, effort, and dollars.

RMS says:

However, the people who wrote the Lucid version didn't try to to make the changes usable for us. They charged ahead without discussing the design with me or with the main Emacs maintainer. They reimplemented things we had already done. I asked them to do some work making parts usable for us, but they said no.

This is not how we at Lucid remember events, although it may be a difference without a distinction. I'll recount the events as I recall them and let the reader supply the characterization.

Lucid set a course 4 years ago to develop a C/C++ programming environment, using Emacs as a key component. We needed a small number of specific things:

• it had to run under X using multiple windows
• it had to support multiple fonts (at least a little)
• it had to support embedded glyphs (icons)
• it had to support attaching active regions to contiguous blocks of characters (to support a hypertext-like facility)
• it had to support menubars
• it had to support readonly regions of text

My goals with respect to FSF were these:

• do anything (including paying money) to get our changes into the official FSF sources
• not sacrifice quality or the schedule (which was lengthy, and therefore apparently safe)

For the first two years we prototyped the functionality we needed in an X-widget-based text editor and in Epoch.

By the Summer of 1990, Emacs 19 appeared to be right around the corner, so we entered discussions with the person at FSF whom we were led to believe was the official Emacs 19 coder - Joe Arceneaux. It appeared that our goals and FSF's were in line, and so he proposed that we pay him a consulting fee to speed up Emacs 19 development plus get our changes in. We agreed, wrote a contract, and volunteered manpower to help.

The contract period started October 1990, was to last approximately 9 months. We were to pay him approximately $50k. He was paid on milestones, on which he appeared to deliver. In addition, we provided him office space and his own Sparcstation at Lucid. (At the conclusion of this period, it appeared that Joe Arceneaux was no longer the official Emacs 19 coder.)

In the Spring of 1991 it became apparent that the effort was going badly - Emacs 19 as a text editor was slow, and full of bugs. The only people at Lucid who dared use it were Arceneaux and myself. Moreover, Emacs 19 performed hideously in the new areas outlined above. Its behavior was especially bad with respect to active regions -- even worse than our seat-of-the-pants prototype in Epoch.

Nevertheless, we persevered: We re-arranged our Emacs team and priorities to work more closely with Arceneaux, and we were confident the arrangement could work. To be safe, I put 4 of our best hackers on it with Arceneaux. By Summer of last year we became aware that RMS had hired/appointed Jim Blandy as the official Emacs 19 person.

At that time, one of our hackers went out to Cambridge to talk RMS about how to proceed, and it appeared we reached a reasonable working arrangement. But, as time went on we determined that the FSF code for Emacs 19 was too unwieldy, bug-ridden, and convoluted to work with, so we started rewriting parts of it.

In addition to not listening to any design ideas we had, RMS's policy was to require us to provide source diffs and to explain every changed line. Because our efforts towards release were mounting, and we were re-writing massive amounts of the system, source diffs didn't make sense, but RMS and Blandy were unwilling or unable to work at a higher level. I traveled to Cambridge to work things out, and my team had numerous phone conversations with RMS.

One area that was completely rewritten was the implementation of active regions. Various attempts were made to discuss the design with RMS, including a 2 hour telephone conversation from one of my hackers, but he was completely unwilling to consider another point of view. At first, he apparently didn't understand the differences between attaching active regions to text and attaching fonts to text, and when he finally did, he was so wedded to his design for putting fonts in the text that he wanted us to implement BOTH his design for fonts and ours for other active region support.

Our hacker wanted RMS to cooperate to the extent that we would first implement our version to support all of these features, and see how well it worked, but he wouldn't even agree to that. It is worth noting that with the original FSF version, in spite of some rather elaborate tree-balancing code and other hackery for precomputing display characteristics, Emacs redisplay spent more than 40% of its time in the active regions code. In our much simpler version it spends significantly less than 1% of the redisplay time in the corresponding code, even for buffers with a few thousand active regions. (This isn't to say that the original design couldn't be made to work, but several man-months of effort had gone into their original version, and it took less than 2 man-months to implement and debug the code that we shipped with our version. Finally, our version is approximately 1/2 the number of lines of the original, it is more general, and it uses less memory.)

Early last Fall, after spending approximately $70k on Arceneaux, we let his contract expire, but we did put a leased 38.4 kb digital line and an X terminal in Arceneaux's apartment in San Francisco (35 miles away from Lucid), so he could continue to work on Emacs 19 for FSF with Lucid support.

When we learned last Summer that FSF had established an Emacs steering committee, and no one from my organization was put on that, or even informed of its existence, we came to the conclusion that the FSF had no interest in working with us, and we stopped trying. The punch line of RMS' statement is true -- we wouldn't do things they way he wanted. But we didn't ``charge ahead'' without talking to FSF. While we did reimplement some things that they had already done, the things we reimplemented were completely broken. It's true that in the end our work has been unusable by the FSF, but that seems to us to have more to do with their willingness to use it than with anything else.

RMS concludes:

The result is that not much of the code in Lucid's version will ever appear in the FSF's version. Some of the features are worth having, but need to be reimplemented. If you'd like this to happen sooner, one way is to volunteer to do some of the work.

I have trouble imagining how anyone could have worked with with them on this. We may have failed in this attempt, but we have a history of trying, and we have succeeded in other areas.

In addition to working on Emacs, and contributing to Cygnus initial implementations of some new features for gdb, such as a malloc that supports multiple heaps, code to save the internal gdb ``symbol table'' state for a given executable file, and a various C++ debugging features, (as well as paying $100K to have these features officially incorporated into gdb), during the past year we dedicated approximately 1 person-year, spread over 3 people, to producing a new Emacs manual. In addition we have spent a lot of time trying to figure out how to work amicably with FSF.

We have also supported FSF and RMS by selecting several FSF components for our product, an endorsement of their quality and usefulness. Since 1987 Lucid has regularly contributed code (and occasionally money) to FSF. Lucid paid over $200k to support FSF products over the last 2 years. We contributed over 8 hacker-years to improve FSF products.

I personally am a member of LPF (not related to FSF, but related to RMS). I travel about debating various people in RMS's name, and I have actively supported RMS in various private ways. I couldn't figure out how to work with FSF and RMS even though I had years to do it, hundreds of thousands of dollars to spend, and 8 man-years to volunteer. What would you have done differently?

We offer a free Emacs 19 to anyone who wants it, under exactly the same conditions as the FSF, providing commercial support to our paying customers (all improvements arising out of this support become part of the free version as well, of course), as well as answering questions and incorporating changes received from users on the net, and we will continue to develop it. We will rewrite the redisplay code, and we will very probably do work to internationalize Emacs. We also are using scarce documentation resources on the Emacs manual (copyright still held jointly with RMS). However, unlike the FSF, we will also listen to ideas from other people. For one thing, if someone implements an improved version of a feature, we aren't arrogant or inflexible enough to refuse to consider using it. The hackers that worked on our version are concerned about doing a good job for the sake of Emacs, not just for the sake of our company or their egos.

By this time I've ceased expecting that we will work closely with FSF on Emacs, but we would still like to do so. As far as we can tell, the reason that FSF won't use any of our work is that they aren't interested in working *with* us (rather they want us to work *for* them, doing precisely what they tell us to do the way they say to do it, and on their own schedule). Perhaps they assume that our designs and implementations are substandard, but as far as I know they have never really looked at what we have done.

In the summer of 1990, I too thought Emacs 19 would come out soon.

One reason it didn't is that Lucid hired away the maintainer and put him to work adding new features that you wanted, instead of getting the program into shape for release. No one discussed with me how this would affect the maintenance of Emacs. In fact, I wasn't even told until a couple of weeks before he was going.

The features Lucid wanted him to implement were useful. But at that time, I wanted to get Emacs 19 out the door--not delay the release for additional features.

I couldn't change the plans, so I had to make the best of them. I suggested a design to him, one oriented toward editing formatted text--a feature I wanted Emacs to have, eventually.

But I had to find someone else to work on getting Emacs 19 ready for release. There was only one person available--Jim Blandy. He is careful and methodical, but not fast, and he hadn't worked on such a large program before. It took him a while to get into the swing of things. Also, he was unhappy to leave his previous project, which was a desktop system.

(To be fair, there have been other causes of delay as well, compounded with these.)

Since then, Lucid has sometimes tried to cooperate with us. But the attempts have not worked.

There have been communication failures. For example, last summer Gabriel found out about an Emacs Steering Committee which no one from Lucid had been invited to join. Apparently I wasn't invited, or even told--I don't recall ever hearing about such a committee. I think something got garbled in transmission.

But if I had set up a committee, and I had looked for who should be on it, I wouldn't have thought of Lucid. I had been told they had stopped employing Arceneaux, but I didn't know they were continuing to work on a version of Emacs. I thought they weren't doing anything.

Last summer I was told that someone there would work on a project that was needed for Emacs--updating the manual, I think. But in September, when I asked him about progress, the he told me he had dropped the project; other higher priority work took all his time.

Later I learned that Lucid had people other than Arceneaux working on a version of Emacs. In December, Gabriel was in this area, and he told me that he wanted Lucid's programming work in Emacs to fit into the FSF Emacs release, and he told me how to reach the programmers there. I was glad to discover a source of additional hands for the work. So I called them, wanting to discuss the design.

But they told that they had already chosen the design and done so much implementation that they wouldn't consider redoing any of it. They had a deadline and had to finish following the path they had set out on.

They had not tried to talk with Jim Blandy to find out what he was doing, working on the Emacs 19 release. I'm not sure they knew he was the person doing it. If I'd known what they were working on, I could have told them they should speak with him.

When people offer the FSF changes to programs that the FSF maintains, the FSF maintainer always has to consider them line by line. That's part of the maintainer's task. And the maintainer might reject some of the changes, or ask for rewriting--that's also part of the maintainer's task.

But mainly we didn't even get to this stage. Problems became visible at the general design level.

During that conversation I found out about the problem with slowness of interval processing. I called Arceneaux and looked at his code, and found a localized bug in the balancing of binary trees. If you created all the intervals from front to back, it would do no balancing, leaving the tree maximally unbalanced. The effect was a rather roundabout linear search. As luck would have it, the Lucid application usually created intervals from front to back, so they always saw the worst possible behavior. Anyway, the bug was simple once found. This all took a few days.

The Lucid programmers had seen the slowness and decided that the basic approach was at fault. So they decided to rewrite the interval code from scratch. They said they didn't need to support editing large files of formatted text; for their usage, a small number of intervals was enough. So they wrote a faster, straightforward linear search.

They were willing to offer FSF the code for this--but it isn't useful. No matter how cleanly written, a linear search becomes a disaster for large files with lots of intervals.

When I explained why the binary tree approach was not inherently slow, they said it was too late--the code worked for their purposes, and because of their deadline, they couldn't consider redoing anything that worked well enough for their needs.

This is what I mean by charging ahead. If they had told me the problem they saw before choosing to start from scratch, I could have told them where the problem was, and they could have made a solid binary tree implementation. But they didn't tell me.

So since then I've been giving Joe Arceneaux more supervision, and I expect he will provide an interval implementation for Emacs 20. Not 19, because I don't want to delay that for these features.

I don't think Gabriel wanted this to happen. I think he had good intentions but didn't explain them to the troops. They did what they thought they were supposed to do, which was to implement a certain set of features--with no particular requirement to discuss the design with me.

I hope that Lucid would like to rectify the situation, but it's entirely in their hands.

during the past year we dedicated approximately 1 person-year, spread over 3 people, to producing a new Emacs manual.

This might be a very useful thing. But why didn't you tell me sooner?

Last summer I was put in touch with someone at Lucid who was going to work on updating the Emacs manual--documenting the new features already implemented, I understood this to mean. (Later he told me he had had to drop the project.) But your message today was the first I had heard of a whole new manual. Is it free? Can I read it?

I'm sorry you feel it is hard to figure out how to work with me. Sometimes I am hard to work with. Sometimes I'm stubborn. But I do manage to work with a lot of other people.

Emacs is full of ideas contributed by others, some of whom contributed the code as well. But you can't expect to bat 1000. Odds are I won't like every single idea you suggest.

If you let me choose a subset of your suggestions, I'll be glad to accept some of them. But if you make me choose all or none, most likely I'll have to choose none. I don't think you meant to do that, but when you don't include me in the plans until too late, that is the effect.

There are some good ideas in what Lucid did. I've already accepted one of them: I've asked Joe Arceneaux to implement temporary highlighting intervals that are not "part of the text", in addition to permanent intervals that are part of it. It was hearing how Lucid was going to use this that made me realize it was important.

But I can't use this idea the easy way, by using Lucid's code, because that code isn't designed to fit in with permanent intervals support. Having temporary intervals is a good idea, but that doesn't make the lack of permanent intervals a good idea.

This is why I would like people to adapt the Lucid code--so that I can combine some of your ideas with the other ideas I want to use.

If you would like to work with me, the first step should be to tell me what sort of thing you want to work on, before you start work. The second step is to discuss the design with me before you commit to a particular design. Then we can consider various alternatives and work out a good combination of ideas.

So since then I've been giving Joe Arceneaux more supervision, and I expect he will provide an interval implementation for Emacs 20. Not 19, because I don't want to delay that for these features.

What I understand from that is that if I want these features now I should continue to use lucid-emacs as my Emacs 19, and wait for a future Emacs 20 which might be some kind of a merge with lucid-emacs, leading to one FSF Emacs.

Danny.

So since then I've been giving Joe Arceneaux more supervision, and I expect he will provide an interval implementation for Emacs 20. Not 19, because I don't want to delay that for these features.

What I understand from that is that if I want these features now I should continue to use lucid-emacs as my Emacs 19, and wait for a future Emacs 20 which might be some kind of a merge with lucid-emacs, leading to one FSF Emacs.

Using the phrase "these features" implies that one set of features is being discussed. That's not really true.(though there is some overlap).

For exampe, Emacs 20 will support including formatting in the files you edit, and also support temporary highlighting intervals. Lucid Emacs supports only the latter. (Of course, our other versions don't support either.)

I read your full answer to RMS and I believe it all.

I would really like to hear FSF's answer.

Let the users community be the judge.

Danny.

Using the phrase "these features" implies that one set of features is being discussed. That's not really true.(though there is some overlap).

Since Emacs 19 was not released yet, I do not have any way to know which of the new features I see in lucid-emacs 19, (new means not in Emacs 18) are from Emacs 19 and which are lucid's features.

If there is a NEWS file for Emacs 19 I'd be happy to compare it with lucid's NEWS file.

Danny.

As the person somewhat in the middle this extended phlegm, it strikes me that both accounts of this history contain a certain degree of inaccuracy.

Be that as it may, I would like to urge the parties involved to forget all this and again make efforts to join forces. Lucid has some talented hackers which can and have done good things for emacs (in fact, version 19 already uses some of their code), and if the "bosses" can work out their politics, the community will benefit.

Joseph Arceneaux

I would like to urge the parties involved to forget all this and again make efforts to join forces.

I heartily agree. I just a mere user but, from the outside, this is starting to look like the Sun/AT&T vs OSF battle for the "One true Unix".

GNU Emacs became a de facto standard because it had features nothing else had and because it was widely available. Right now, it seems that Lucid Emacs has those two qualities while GNU Emacs, alas, isn't available. Please! Think of the community and the delays that would be introduced by the confusion. Let's face it; Lucid is not going to drop their version since sales depend on it. On the other hand, the FSF is not satisfied with it and is going to redo some coding and implement other features.

Consider this suggestion: let's all work towards making Lucid Emacs the long awaited "official" Emacs 19, learn from it, and integrate the FSF's long term plans into Emacs 20.

Thanks to Kyle Jones, we now have a pair of newsgroups bidirectionally gatewayed into the bug-lucid-emacs and help-lucid-emacs newsgroups. They are alt.lucid-emacs.bug and alt.lucid-emacs.help.

I expect that many of you will want to unsubscribe from the mailing lists, and read the messages in newsgroup form instead. However, I would advise you to wait a few weeks before unsubscribing, so that we know that the gateway is functioning properly.

We wanted the newsgroups to be gnu.lucid-emacs.bug and gnu.lucid-emacs.help, but the FSF folks are not willing to allow the creation of those groups. In fact, it would appear that they are unwilling even to allow us to announce the existence of our version of emacs on their newsgroups.

-- Jamie

Hans Muller writes:

We're considering making a committment to Epoch or Lucid emacs and I was curious about why you decided to make the switch - beyond the fact that the hilit package makes the two comparable. Are there things about Lucid emacs you prefer over Epoch? Any thoughts you have on the subject would be appreciated.

Hans,

I made the switch from emacs 18.57 to epoch 3.2 a while back, the biggest reason being the multiple screens from one process. Having the ability to edit multiple buffers of text, and have more than one on the display at once was very important. One other consideration in making the switch was region hilighting via a mouse drag. As more people started writing code for epoch, features like displaying a filename at the point in another screen, man pages in another screen with hyper-text like See Also sections, a nifty info-mode and so on became part of my .emacs. Epoch gave the the x-support that seemed intuitive to have in this environment. One the source code hilighting came out, I was hooked. I cannot live without it. It is the one thing that kept me from moving to lemacs sooner. The reason I wanted to stop using Epoch was it has some intermittent bugs, like spontaneous quitting, killing random xterms when it did quit, and just an overall level of unstableness, not unusable, just annoying.

Lemacs probably has a similar level of annoyance, but it also has some very nice features. I like the menubar (alot). I was using the bam package (I forgot by whom) when I was using epoch. It gave me the same functionality the the menu bar in lemacs does. The hilighting package in several time faster under lemacs than under epoch (although they are different packages, hilit vs brightlight, I get the same functionality at 1/4 the cost) The interface into x with lemacs is much cleaner, I get, out of the box, functionality that one would expect in an x-based editor that would take several packages under epoch. Things like double and triple clicks for word and line hilighting and the menubar. Although I am not an elisp programmer, the overall interface to lemacs is much cleaner than with epoch. It makes (sorry guys) epoch look like a hack.

Just my $.02.

Hope this helps,

Bob

I made the switch from emacs 18.57 to epoch 3.2 a while back ... The reason I wanted to stop using Epoch was it has some intermittent bugs, like spontaneous quitting, killing random xterms when it did quit, and just an overall level of unstableness, not unusable, just annoying.

Umm, folks, Epoch 4.0 has been out for quite a while now and is quite a bit more stable than 3.2 was. Spontaneous quitting, killing random xterms, et al certainly ought to all be fixed in 4.0 (if they're not, PLEASE let me know).

Thanks,

Marc

-- Marc ``Epoch is my Middle Name'' Andreessen
marca@ncsa.uiuc.edu

People should keep in mind that the successor of Emacs 18 does not come from Lucid. Its is GNU Emacs 19. I still can't say exactly when public release is going to be, but we are about to start testing at a number of sites.

GNU Emacs 19 will support a broad spectrum of machines, like Emacs 18. Ensuring this is the purpose of the testing we are about to do.

(Please don't volunteer; we have enough pretesters, and if more people offer, dealing with those messages will slow things down.)

Once we make sure it is indeed working reliably on various different systems, we will have a public beta test release.

Emacs 19 does support adding properties to ranges of text, and using these to switch fonts. In the future, the Epoch people will help merge support for variable-width fonts.

Meanwhile, I have almost finished updating the Emacs Lisp manual. Its next edition will describe Emacs 19. There will be an announcement when we know when this edition will be available.

People should keep in mind that the successor of Emacs 18 does not come from Lucid. Its is GNU Emacs 19. I still can't say exactly when public release is going to be, but we are about to start testing at a number of sites.

Free software is a great thing - freedom of distribution and use, and freedom from price. It's such a great thing that some companies recognize the value of giving things away, as a mechanism of goodwill, of ``paying back'' the community it serves, and as a mechanism to show potential customers the quality and style of software you can expect from those companies.

Four years ago we embarked on a development project one of whose central portions would be Emacs, but an Emacs with enhanced capabilities - active regions, properties of text, fonts, color, menubars, pop-up menus, and true X integration. Free Software Foundation at the time was beginning to work on Emacs 19. Lucid paid the author of the then-current Emacs 19 prototype to help us get it ready. Well, $50,000 poorer and 18 months older, we were no closer to our goal than when we started.

So, I assigned 5 of our best developers to work on Emacs 19, and last summer we released it to the world. It runs on a bunch of platforms, it is actively maintained (it is the centerpiece of our user interface for our commercial product), thousands of people use it, we are working with volunteers around the world, and - the best part - we operate just like FSF on it: it's free, source code and all, there's an updated manual for it, and we incorporate all reasonable changes just the way FSF does.

The difference is, we have Emacs 19, it is well-tested, and its users love it. Another difference is, we have 35 developers and we don't need handouts to keep maintaining Emacs - it's just good business.

The concept of free software doesn't belong to anyone. The dream of those who value free software is that the idea will be so compelling that others will pick up the standard and improve it, that others will take more time, do more things, champion the cause so that the community get more because fresh blood is applied to it.

We waited for years and years for Emacs 19. The idea of free software is a good one, and if you cannot live up to its needs, you should let go. Free software is an idea so good that you shouldn't smother it by claiming you own it. Free Software Foundation doesn't own the concept of free software, and it doesn't own the exclusive rights to work on Emacs.

Wait for the private tests, wait for the beta tests, wait for general release, wait, wait, wait. Or, FTP it from /pub/gnu/lucid on labrea.stanford.edu today and use it this afternoon. Help us move on from Emacs 19 to Emacs 20.

-rpg-

Over two years ago, you hired the FSF staff person who I had charged with getting Emacs 19 out the door soon with the features it had then. I got about two weeks notice that he was leaving.

While you set him to work implementing new features that I had planned for Emacs 20, I had to try to get Emacs 19 back on track. It took a while for the new maintainer to come up to speed. The result: a long delay. Emacs 19 would have come out two years ago if not for this.

Later you assigned programmers to work on another version of Emacs without telling me about it. I found out months later by luck. Why didn't you tell me? Why didn't you ask them to discuss the design with me before they implemented it? Whatever the reason, you didn't, which was not a very cooperative way of working.

Well, we have finally dealt with the problem caused two years ago, and what happens? Now you are asking people to make up their minds that GNU Emacs 19 is dead, quickly, before it has a chance to crack the shell.

Overall your conduct leaves something to be desired.

I would be happy to see someone capable take over the work on Emacs from the FSF. I might have been happy to see Lucid do so, if you had approached it openly and with a good spirit.

Lucid can implement things. And some of them are good features. But some of them are clunky. If you had asked the programmers to work with me, I think it would be better. (People will soon be able to compare the design of analogous features and form their own opinions.)

Above all, you've shown a bad spirit. I want Emacs to be in more worthy hands than that. I think there are possibilities.

I have just about finished updating the Emacs Lisp manual. In the next three weeks I'll update the user's manual.

I went back and found the mail exchange that you and I had about Lucid's involement with Emacs 19 and FSF. I still think we tried as hard as any reasonable organization can to work with you and FSF. I am still saddened and puzzled by your peculiar memory of it. I reproduced the exchange below so people can see it (again) and judge for themselves our relative behavior.

Our Emacs is used widely and I have yet to hear anything but embarassingly high praise for it. We only want to help the community, but I think you just couldn't let go enough to allow us to help. There are things that I think need to be done to it to make it better, but unfortunately you and I continue to duplicate effort. Everyone loves a race, I suppose.

-rpg-

[ ... inclusion of earlier messages elided ... ]

Earlier I did not mention in public how you contributed to the delay in Emacs 19, because I was being polite. (Yes, I really do that sometimes.) I took the blame on myself for the sake of amity.

But now that you are using our mailing lists to actively urge people not to work with us, I have the feeling that it is too late to seek amity. What you tell the public does not match what you actually do.

but I think you just couldn't let go enough to allow us to help.

This is a handy cheap shot. It is always easy to make this accusation, and no one can ever disprove it.

But I can point out that

1. I gave the work on Emacs to other people within the FSF,
2. I've given maintenance on other programs I wrote to people outside the FSF, and
3. I'm raising funds now to give away the maintenance of GCC to someone outside the FSF.
4. I politely kept quiet last year about your role in the Emacs 19 delay, even when you were criticising me publicly, in the hope of achieving cooperation with you.

I wouldn't have done these things if I were the sort of person whose ego would never let go of a project. I hope I will find someone suitable to hand over Emacs maintenance to. Someone competent, cooperative and upright.

I don't think I've found someone yet. You would be capable enough if you were sometimes willing to listen. But at every step of the way you have given cooperation only lip service. It takes special gall to present my earlier politeness as indication I am "misremembering" what happened.

I believe in cooperating with people who approach me trying to cooperate. At one point I hoped you were that person. But at this point, your actions say no.

To the users:

I'm not going to ask you to stop cooperating with Lucid. Since Lucid Emacs is free software, it is not wrong for you to cooperate with them about it.

But I do appeal to you of GNU Emacs to continue working with the FSF on GNU Emacs, and not to listen when Lucid asks you to stop.

It's always fun talking with you, Richard. Listening is a skill many of us could stand to learn. I'll try to learn it if that's what you think my problem is. Shall we try an experiment? I am ready and more than eager to try to make this work one more time - I've got over a million $ into Emacs already. Shall we cc our conversations on the design of our cooperative efforts to this list?

Tell me and JWZ what it takes to start over.

-rpg-

Shall we try an experiment? I am ready and more than eager to try to make this work one more time - I've got over a million $ into Emacs already.

My understanding is that it is technically very hard to merge the two versions now.

Lucid Emacs is set up to use and require an X toolkit; our version does not use one. I think it is too expensive to require a toolkit. Especially since we have already got most of the features we want, without a toolkit.

In principle, I like the idea of supporting both modes of operation--with toolkit and without. But the programmers at Lucid told me last summer that using a toolkit required far-reaching changes in many parts of Emacs, including making C-g fail to interrupt a running program (something I worked hard a couple of years ago to make work right).

So it sounds impractical to support both modes--at least, unless something is done to change the requirements of the toolkit.

I don't blame Lucid for this. If anyone made a mistake, it was those who designed the X toolkits.

But here is an idea. You mainly support Suns. They have threads. More and more other systems have threads.

Would it be possible to use a separate thread to do the synchronous interaction with the toolkit, and make it mimic the usual kind of asynchronous input interface Emacs expects? And hide that within a toolkit interface file that fits into GNU Emacs 19 in a modular way? This might also make C-g work right in the toolkit case.

Perhaps this will make it possible after all to support both toolkit and non-toolkit modes of operation. Perhaps then our versions could be merged.

Two other areas of difference are in the design of menus and of events. Lucid Emacs has separate opaque data types for these. GNU Emacs uses pre-existing Lisp objects for the same job: keymaps serve as menus, and ordinary numbers, symbols and lists serve as input events. It should not be very hard to write Lisp functions to imitate the Lucid functions, using these data structures, and then support both interfaces.

A final area of difference is in intervals. But if that is the only discrepancy, I'm sure it won't be hard to deal with.

Richard Stallman wrote:

Lucid Emacs is set up to use and require an X toolkit; our version does not use one. I think it is too expensive to require a toolkit. Especially since we have already got most of the features we want, without a toolkit.

There are features that you don't have, such as guaranteed correct interaction with X resources and app-defaults files, and the ability to build other widgets into the emacs process.

Also, using a toolkit improves portability, since whoever ported the toolkit has done all of the hairy select() hacking for you already. (But if you want to compile without support for X (which is different from running without X) then this is not an issue, because you have to do this system-dependent hacking anyway.)

I think the overhead of using Xt is not anything to be concerned about; the speed cost associated with Xt is insignificant, and the increase in executable size is both A: small, and nonexistent if you are using shared libraries, and B: proven to be acceptable by the already widespread use of this toolkit and toolkits derived from it.

In principle, I like the idea of supporting both modes of operation--with toolkit and without. But the programmers at Lucid told me last summer that using a toolkit required far-reaching changes in many parts of Emacs, including making C-g fail to interrupt a running program (something I worked hard a couple of years ago to make work right).

The event loop was rewritten to support the Xt-based command loop, true; but it was rewritten in what I believe is a much more retargetable way. It is not dependent on Xt; it just happens that currently Xt is the only backend supported. I'm sure it would be much easier to write a home-grown non-Xt backend for the event-loop with lemacs than it would be to make the FSF v19 command loop work with Xt. (In fact, that's why I designed it this way.)

Perhaps I was unclear in a previous message, but you seem to be confusing an implementation detail of the C-g handing with the user-visible behavior. I spent a lot of time ensuring that the emacs command loop, including its handling of C-g as an elisp-level interrupt, is identical in behavior to emacs 18 from the user's point of view. We would certainly not break things badly enough that C-g didn't work!

I don't blame Lucid for this. If anyone made a mistake, it was those who designed the X toolkits.

Mistakes were made in the design of the X toolkits, to be sure. But I don't think they have adversely affected Lucid Emacs (aside from the time I had to spend to work around them.)

-- Jamie

There is a segment of the community that wants an Emacs that uses a toolkit. The people who use Lucid Emacs are part of that segment.

There is also a segment of people whose machines won't support that. I think there are a large number of such sites with limited memory and no shared libraries, now using GNU Emacs. I know of some myself. I also know I have always had complaints about how large Emacs is, from people whose machines are small and overloaded.

Lucid won't get many complaints from these people. (If you don't like liver, you won't complain to the chef that the chopped liver has liver in it.) But that doesn't mean they are few in number.

There is also a segment of users whose machines don't have X windows.

If all three segments are large, a program that appeals only to one of them can get an enthusiastic reception. I think that's what Lucid has found. But if the FSF follows suit, a large number of users will be unhappy.

So I would like add the capability to use a toolkit, without giving up the ability to work without one. I'd like to add this in a modular way, for the sake of future maintenance.

My beliefs and opinions about Lucid's changes in input structure are all based on what Lucid's staff told me last summer. I was told the toolkit insisted on calling Emacs as a subroutine, and would not deliver another event until Emacs returned to it. I recall noting out that this would stop C-g from working, and that someone confirmed this. I asked if the toolkit provided any way to peek ahead, and was told no.

This information dissuaded me from the goal of merging the two versions. Other Lucid programmers agreed that it was impossible to support both modes of operation because the changes were too pervasive.

Now I think you are saying it is possible, and that you have a clean design to propose. I'm glad to hear it.

Can you write up a technical proposal for how to change the Emacs command loop and/or input reading routines in preparation for supporting toolkit, non-toolkit X, and no-X-at-all modes of operation?

Richard Stallman wrote:

My beliefs and opinions about Lucid's changes in input structure are all based on what Lucid's staff told me last summer. I was told the toolkit insisted on calling Emacs as a subroutine, and would not deliver another event until Emacs returned to it. I recall noting out that this would stop C-g from working, and that someone confirmed this. I asked if the toolkit provided any way to peek ahead, and was told no.

Who told you this? If you think it was me, then we had a very serious miscomunication, because none of this is true, and I have never believed it to be true. And I'm the one who wrote basically all of the code in question.

Can you write up a technical proposal for how to change the Emacs command loop and/or input reading routines in preparation for supporting toolkit, non-toolkit X, and no-X-at-all modes of operation?

Even better, I can point you at a working implementation, liberally commented. I find it upsetting that, despite all of the criticism you heap upon us, it is clear that you have never even *run* Lucid Emacs.

Here is a description of what Lucid Emacs currently does at the lowest level:

[ ... various technical details elided ... ]

My beliefs and opinions about Lucid's changes in input structure are all based on what Lucid's staff told me last summer. I was told the insisted on calling Emacs as a subroutine, and would not deliver another event until Emacs returned to it. I recall noting out that this would stop C-g from working, and that someone confirmed this. I asked if the toolkit provided any way to peek ahead, and was told no.

Who told you this?

I don't know which person at Lucid told me this. But that doesn't seem important. What matters is that it isn't so. I'm glad, because it was discouraging news at the time.

The event_stream mechanism sounds good. Could you send me the code for that part of Lucid Emacs? I think we can install it, if you will do the job of separating it out from the mass of the rest. (This is something you can do much more easily than I.) I doubt it can be installed verbatim, since we've probably made changes in the surrounding context, but at least it sounds like a job worth doing.

Where else are changes needed to work with your toolkit-use code?

Richard Stallman wrote:

The event_stream mechanism sounds good. Could you send me the code for that part of Lucid Emacs? I think we can install it, if you will do the job of separating it out from the mass of the rest. (This is something you can do much more easily than I.) I doubt it can be installed verbatim, since we've probably made changes in the surrounding context, but at least it sounds like a job worth doing.

I'll send you the files, but I doubt there is much surrounding context left at all.

Where else are changes needed to work with your toolkit-use code?

It's hard to say; our changes have been pervasive. As you said, a merge will be a very large task.

-- Jamie

Where else are changes needed to work with your toolkit-use code?

It's hard to say; our changes have been pervasive. As you said, a merge will be a very large task.

It's almost impossible to merge one program with another as a unit, when there is a lot of difference. By far the easiest way is to find cohesive "unit changes" in one program--various parts that were changed together and need each other to work, but are separate from other changes--and think about them one at a time.

The merging in of the changes is most efficiently done by us; the extraction is most efficiently done by you.

Please help us merge by doing this.

What part of the job of producing a merged version is Lucid willing to do?

If the work of merging is up to the FSF, then I can say how we will do it. We'll try to make a version that has all the capabilities of both versions, with as little work as possible. In many cases, we have both implemented pretty much equivalent functionality; we will stick with our implementations of these, while merging in capabilities that we don't have. (These mainly relate to Xt use, and things needed to support it.)

Once such a merged version exists, what would happen then? Will you work on that version, together with us?

I hope so. If you keep working on new features in the current Lucid version, each feature will require considerable rewriting to fit into the merged version. That way the job of merging would be unending.

To put it differently: IF someone contributed a clean optional XView backend, would you then consider integrating it in a future version of GNU Emacs?

Yes. Whether to install it would depend on things like how clean it is, and how much continuing work it would need.

If GNU Emacs will provide a clean separation between backends, the maintenance required for the Notifier backend would be at the level required by the Xt backend.

Supporting a XView GUI (menus, dialog boxes) should be as much work as supporting a widget based GUI, again given a clean generic interface to the GUI front end.

I guess there is no reason to start thinking about integrating XView in GNU Emacs before those interfaces have been defined.

I guess there is no reason to start thinking about integrating XView in GNU Emacs before those interfaces have been defined.

I think we will adopt the event-stream data structure and conventions that Lucid uses, for communication with toolkits of any sort.

rms@gnu.ai.mit.edu (Richard Stallman) writes:

I think we will adopt the event-stream data structure and conventions that Lucid uses, for communication with toolkits of any sort.

That should make it possible to prototype the functionality with Lucid Emacs.

Would the backend be an Emacs 19, Emacs 19.xx or Emacs 20 kind of thing?

How about generic GUI support (menus, dialog boxes)?

We tried very hard to make the event-stream model general enough to support other window systems, and to keep the X-specific code well isolated. There are currently a lot of Xisms in lemacs, but they're almost entirely related to redisplay, which we haven't done much work on yet.

I've always thought that a native NeXTstep port would be great, because I think NeXTstep is a nice window system. Certainly the design won't have been truly vindicated until it is used with a window system other than X. But I suspect that a Windows NT port will be more useful...

And we could also write wrapper functions for the next which implemented whatever X calls emacs was making.

I know of someone who has done this (simulate Xt) for another window system. I think it's a pretty poor way to go about it, though it might be easier in the short term.

However, I wonder how gross and low level these calls are. If they are things like moveXY(x,y); drawString(font, "string"); we could do those easily.

Lemacs doesn't exercise Xlib very heavily (there are probably some abstractions missing, like getting the pixel-size of a string in a particular font, but that's easy to fix), and all of the Xt stuff is isolated under the event-stream callbacks.

-- Jamie

There is also a segment of users whose machines don't have X windows.

Does this imply that FSF's emacs-19 might support other windowing systems?

I was talking about people with no window system.

In principle, we could support other window systems. In practice, we don't have the resources to do so--sorry. It would take a lot of work and would in no way enhance the GNU system.

It's almost impossible to merge one program with another as a unit, when there is a lot of difference. By far the easiest way is to find cohesive "unit changes" in one program--various parts that were changed together and need each other to work, but are separate from other changes--and think about them one at a time.

The merging in of the changes is most efficiently done by us; the extraction is most efficiently done by you.

Please help us merge by doing this.

I've largely merged FSF Emacs V19 into Lucid Emacs (except for redisplay code, keyboard.c, and the neverending changing extra arguments to functions in window.c)

I am totally uninterested in doing the merge the other way, if for no other reason than that I believe the people at Lucid working on Emacs are far more competent and more far tasteful programmers, and aren't wasting large amounts of FSF resources which would much, much better have been spent elsewhere.

Working on programs with people like Jamie Zawinski (or many other excellent hacker types) is also vastly easier than trying to cooperate with RMS on anything.

Our concerns about merging Lucid Emacs with FSF v19 are that there is insufficient motivation on the part of FSF to compromise. In particular, we are concerned that FSF will decide not to include features that we consider critical, or that those features will have an interface that is quite different from what we need and are implemented in a way that preclude supporting the interface that we need.

However, if there was a version of emacs from the FSF that did everything that we require from Lucid Emacs, then we would use it. Also, if v19 has a better interface to something than what we designed, we would be motivated to use it. (In general, a change in the interfaces will make the merge harder, and make it take longer, but it doesn't preclude our switching. An implementation or interface that eliminated functionality important to us would preclude a merge.)

Here are the things (that we can think of at the moment) that we believe we need in a merged version, in no particular order:

• We don't want to lose our tight integration with the Xt event loop, so that we can link arbitrary widgets into emacs.
• We want our implementation of keymaps to be used: we want them to be an abstract data type, not something like "if the third element of the alist is a cons whose car is a vector of length 7, then it represents an aliased indirection into the sixth element of the alist..." (NOTE: Having support for "secondary" data types, similar to structure subtypes in Common Lisp, would be quite acceptable, but alists are not.)
• We don't want any ASCII limitations: all X chords should be bindable. We want our extensions to the syntax for the define-key function to work.
• We want input and display of all ISO-8859-1 characters.
• We want ICCCM compliance.
• We want a menubar, popup menus, and dialog boxes, and the ability to use the native toolkit widgets to implement it to guarantee that they comply with local look-n-feel customs.
• We want the menubar to be buffer-local, and update efficiently.

  We want the menubar to display the keybindings corresponding to the commands listed in it, and update this dynamically.

• We want to avoid compiling pathnames into the emacs executable: it should figure out where things are at run-time based on the location of the executable.
• We do not want menus implemented in terms of keymaps or anything odd like that. This sort of false generality just makes things harder to maintain.
• We want our implementation of extents and faces, or something with a superset of the functionality and performance characteristics.
• We want zmacs-style region highlighting.

I'm glad that you would be willing to use a merged version *if* the FSF does all the work of producing it.

But this sounds like a statement that Lucid is not willing to do any of the work necessary to produce a merged version. I'm sad to hear that. At the same time, Lucid sounds very exigent regarding what features we should include and exclude.

Thus, we have a combination of stringent demands, with a refusal to help accomplish them. The combination sets up a situation where cooperation is predictably unlikely. We will certainly merge in the features that seem important (those we don't have already), but whether we merge in every last thing Lucid wants, depends on our resources and on how generally useful the features seem to us.

We can install more of the features Lucid wants if Lucid helps do the job.

A couple of specific points are disturbing in another way: they seem to insist that we remove some of the features of GNU Emacs 19--such as, its unification of menus with keymaps, and its use of standard Lisp data types rather than creating new opaque types.

Is there any chance you could accept a compromise? Such as support for the interfaces that you now use, as well as the ones we prefer?

RMS wrote:

But this sounds like a statement that Lucid is not willing to do any of the work necessary to produce a merged version.

This is not the case; we are willing to help work towards a merge. We are not willing to do it ourselves unless you say that we get to be the final arbiter of what goes in. Without that, we do not, at this time, have any confidence that we won't be wasting our time.

The facts speak for themselves: we have tried very hard to work with the FSF. We have spent hundreds of thousands of dollars and several man-years on it. Clearly we have not done so successfully, regardless of where the fault for the failure lies, but we did try, that is undeniable.

We don't think it's too much to ask for you to make the first move as a show of faith. We've been burned before. We'd like to see some commitment from your side before committing any more money or resources to this endeavor.

At the same time, Lucid sounds very exigent regarding what features we should include and exclude.

Whether you include or exclude them from your version of emacs is your decision. But these are features we need, and if they are not in your version of emacs, then your version of emacs will simply not be adequate for our purposes.

A couple of specific points are disturbing in another way: they seem to insist that we remove some of the features of GNU Emacs 19--such as, its unification of menus with keymaps, and its use of standard Lisp data types rather than creating new opaque types.

You call them features, we call them bugs. We have contractual obligations to maintain the version of emacs that we distribute, and the design of the data structures used has a great bearing on how difficult that is.

Is there any chance you could accept a compromise? Such as support for the interfaces that you now use, as well as the ones we prefer?

Yes, compromise is always possible.

-- Jamie

Working on a version of Emacs and working with the FSF are not identical. In the past, Lucid has done a lot of work on Lucid Emacs, but only rarely tried to work with the FSF.

Your latest message states the intention to do some work on merging in the future, which is good. The emotional tone, shows a contrary attitude. I'm not sure what to make of this mixed message.

The FSF will merge in some features of Lucid Emacs in any case, now that I see they are reasonable to use. Specifically, those that pertain to generic interfaces to window-system support code, followed by the support for an X toolkit as a compile-time option.

In principle, there's no capability that we would object to supporting in some fashion. Some we would do the work to add, but some we would not (if we think it is of minor importance or that we have a better capability already). Thus, merging of the latter class depends on what Lucid does.

We seem to disagree about how menus should be supported. I think it is clean, simple and convenient to have keymaps serve also as menus. You think an abstract interface is cleaner.

If we are limited to choosing between one of two alternatives, this looks like an irreconcilable difference.

I've proposed one compromise--supporting your interface and ours--but you don't like that.

Can you please propose another?

I need to study your implementation of menus and keymaps. I am away all next week.

My worry is that the two abstractions should not be mixed up unless there is a fundamentally good reason to. Implementation ease or implementation shortcut is not a reason for me. But, I'll look at it.

-rpg-

Regarding the lucid merge and the use windowing toolkits:

As an avid Epoch and NeXTstep user, I'd be interested in hearing more about the possibility of generalizing the windowing interface of Emacs 19.

I'm not familiar with Lucid Emacs, so I don't have a feel for whether this will be necessary, practical, or even desirable.

From rms/lucid discussion, it wasn't entirely clear whether or not supporting both the vanilla X & Xt toolkit designs would require enough restructuring to make another layer of indirection worthwhile.

I'd like to work on native NeXTstep support (if such an interface could be implemented).

marcus daniels

Based on what I've heard in the recent discussion, it ought to be possible for GNU Emacs to support Xt in the future as an optional feature. The first step towards this will be supporting a generic interface for reading input events. GNU Emacs 19 already has a generic event structure, much like the Lucid one, so the main job is supporting general hooks for interface to the program that does the reading.

The second step might be to support a new kind of frame that interfaces via Xt instead of via Xlib, and perhaps to create a set of clean hooks for that interfacing.

From: jimb@totoro.cs.oberlin.edu (Jim Blandy)
Date: Mon, 3 May 1993 15:35:50 -0700

Does this in any way imply that a merge of the two versions is getting any closer?

GNU Emacs incoroporates many of the features of Lucid Emacs. We are using sections of Lucid's code, and have benefited from their advice.

We have a Lucid emulation package (lucid.el) which may or may not be terribly effective; we'll have to see it in use.

The FSF's "lucid.el" consists largely of

• aliases of functions named "*screen*" in Lucid Emacs and Emacs 18 but renamed "*frame*" in FSF 19 (this is a desirable name change, but I believe the Lucid Emacs authors preferred compatibility over better nomenclature at the time.) I imagine that Lucid may change also, given a precedent for user pain.
• a couple of utility Lisp functions which RMS doesn't want in his small, pure, simple Emacs (eg copy-tree, remove-hook, remprop)
• some functions which it is totally insane not to include as part of basic Emacs (keymap-parent, map-keymap, etc) "What's data abstraction anyway?"
• a couple of miscellaneous utilities (eg add-timeout)

Not anything really effective on the whole, I'd imagine, to deal with the real incompatibilities (the event-stream, faces and extents.) As far as I can see, RMS' whole course has been to make everything to do with the real issues as divergent as possible by systematically substituting worse for better.

Some features of Lucid we will probably never have; Stallman doesn't like some of them. Since the people with real influence over the designs of the products (Zawinski, Stallman) are both rather stubborn, I wouldn't hold my breath for a complete unification.

My aim wasn't a "unification", either because the RMSmacs code is simply perverse and wrong in many areas (eg events, keymaps) or because it has diverged (deliberately) too far from Lucid's code for a simple effort to be made by a non-window-system kind of guy (faces, screens, etc.)

My aim was to incorporate as many compatible changes made in the FSF code (many bug fixes, some features, many dubious kludges like find-file-name-handler) as possible into the exiting Lucid Emacs distribution, both to take advantage of good work done by FSF Emacs hackers and to make all future source-compare-and-merge operations simpler (if anybody has the resolve, technical ability and time to perform them).

[Various people listing reasons for not switching to lemacs...]

You might add horizontal scrolling and splitting to the list.

Several people have bashed GNU and RMS lately about their alleged disregard for established precedent. Just to set the record straight, the Lucid folks are no less guilty of this particular sin. Lucid threw out RMS's event and keymap model, for example. Maybe Lucid's approach was better (not entirely clear; each system has plusses and minusses). But RMS was there first, and just because Lucid Emacs was released earlier is no reason to fault RMS for sticking to his original scheme.

GNU Emacs 19 makes a much greater attempt at backward compatibility. One little thing I noticed recently is that the unread-command-char variable is still supported, *as well as* an alternative that holds events instead of chars. Lucid simply deleted the old variable, and it was a giant pain for me to fix all the references to it in Calc. I sure wish Lucid had put compatibility at a higher priority.

-- Dave

Dave Gillespie wrote:

Lucid threw out RMS's event and keymap model, for example. Maybe Lucid's approach was better (not entirely clear; each system has plusses and minusses).

What do you think is better about RMS's event and keymap model, beyond the fact that it was there first (i.e., compatibility)?

One little thing I noticed recently is that the unread-command-char variable is still supported, *as well as* an alternative that holds events instead of chars. Lucid simply deleted the old variable, and it was a giant pain for me to fix all the references to it in Calc. I sure wish Lucid had put compatibility at a higher priority.

The new event model in Lucid Emacs is a portability problem, to be sure. I did what I could to remain compatible with the old model (keeping last-command-char as the ASCII equivalent of last-command-event, for example.) But I could not think of a way to keep unread-command-char and still *make progress*.

My goal was to make emacs a better editor than it was before. Perhaps you don't think that removing the restriction to the ASCII character set was worth the pain; I think it was.

If you don't like the design decisions I made, please suggest better ones. Converting to ASCII and back loses information. If a package were to read and unread a character as ASCII, it could easily cause the wrong command to be executed. I could put this hack in, and it would mostly work most of the time, but it's the wrong thing. There are arguments in favor of improving software by adding kludge on top of kludge instead of stepping back and reevaluating the original decisions. I chose not to take that approach. Maybe you disagree. But I gave you an editor more than a year ago that I think was a vast improvement over what I started with.

-- Jamie

What do you think is better about RMS's event and keymap model, beyond the fact that it was there first (i.e., compatibility)?

One: Simple keystroke events are represented the same as they were before, which goes a long way toward compatibility with Emacs 18. The mapping between strings and event vectors is much more natural, leading GNU Emacs 19 to provide still more compatibility in that area. Keymaps, while not backward-compatible, are fairly close.

Two: The most common events are atomic objects, which means he doesn't need to resort to explicit memory management of event objects to keep from garbage collecting all day long. The bulk of events don't need to be collected because they're just symbols or integers.

Three: Events and keymaps are built from plain Lisp objects, which makes them easier to examine during debugging, and easier to construct and modify if necessary.

Four: Events are more usage-oriented, less X-oriented. This will probably make for an easier transition to other window systems in the future.

Those are the things inherent to the representation that I like. He also has some features that you don't have, but that you could add without losing your event model. The handling of down, up, and drag events is nice. The Lucid model acts on the click, whereas RMS's model usually acts on the release. This seems to work fine in practice and allows him to generate drag events for the user. (Although he doesn't yet, he could also generate double-click events on the second of two consecutive clicks---hint hint, nudge nudge.) It also fixes the stupid problem where mouse commands leave a message behind only to have it erased when the release-event is processed.

On the other hand, your method has its own advantages:

A: Keystroke events carry position and timestamp information, which allows keys to emulate extra mouse buttons in a pinch.

B: All events have the same representation, which has a certain elegance to it. (This elegance is muffled by the need to keep the garbage collector away from event management, though.)

C: There was something else I liked, but I can't think of it offhand. It will come back to me eventually.

Overall though, I seem to have found more items in RMS' favor. Before I made the list, I was thinking of the two systems as about equal in merits.

But I could not think of a way to keep unread-command-char and still *make progress*.

GNU Emacs simply has both. If there is a character in unread-command-char, it stuffs it into the input stream, and if there are events in unread-command-events, it stuffs those, too. There is some order of priority, but it doesn't really matter.

You could still have retained compatibility by using the name "unread-command-char" instead of "unread-command-event" for your current variable, and allowing it to contain integers as well as event objects.

My goal was to make emacs a better editor than it was before. Perhaps you don't think that removing the restriction to the ASCII character set was worth the pain; I think it was.

Maybe RMS' system has evolved since you first saw it. He represents function keys as symbols, and he can also represent a full set of modifier bits on printable keys. In terms of keys themselves, both systems are equally expressive as far as I can see. Yours does include time stamps, though, which could conceivably be handy. (Not enough to choose them over the simplicity of RMS' model, though, in my opinion.)

If a package were to read and unread a character as ASCII, it could easily cause the wrong command to be executed.

Yes, but there are about a zillion places in Calc where I stuff an event that I *know* is a printable character, and I had to change every single one of these.

Your model does not detect incompatible code, either, because older programs can simply go ahead and store something in "unread-command-char" and then silently fail.

But I gave you an editor more than a year ago that I think was a vast improvement over what I started with.

No question about it.

-- Dave

Dave Gillespie wrote:

What do you think is better about RMS's event and keymap model, beyond the fact that it was there first (i.e., compatibility)?

One: Simple keystroke events are represented the same as they were before, which goes a long way toward compatibility with Emacs 18.

Yes, I realize this; that's why I said "beyond compatibility." I wanted to know what you meant besides this.

The mapping between strings and event vectors is much more natural, leading GNU Emacs 19 to provide still more compatibility in that area.

Keys and buffer-characters are not the same thing. Strings are made of the stuff that buffers are made of. Keys are not. Show me how to represent Control-Shift-A or Control-Backspace in a string.

Two: The most common events are atomic objects, which means he doesn't need to resort to explicit memory management of event objects to keep from garbage collecting all day long.

You don't need to resort to explicit memory management. You can drop the events on the floor and they will be garbage collected like any other lisp object. It's simply more efficient to reuse them (this probably only matters in one place in all of emacs: the single event that is reused by the main command loop; and even there it hardly matters at all.)

It such a common misconception that you *need* to explicitly manage events that I'm starting to regret that I made it possible at all...

Three: Events and keymaps are built from plain Lisp objects, which makes them easier to examine during debugging, and easier to construct and modify if necessary.

And harder to programmatically tell apart from other things ("eventp" is unpleasantly heuristic.) But perhaps they should be less opaque; give me examples where you've wanted this.

Four: Events are more usage-oriented, less X-oriented. This will probably make for an easier transition to other window systems in the future.

I very carefully designed the event structures to expose only those components that emacs would need when running under *any* window system. They are not X specific at all; why do you say they are?

The Lucid model acts on the click, whereas RMS's model usually acts on the release.

Lucid delivers both up and down events, and you can bind commands to either.

This seems to work fine in practice and allows him to generate drag events for the user. (Although he doesn't yet, he could also generate double-click events on the second of two consecutive clicks---hint hint, nudge nudge.)

Handling of double click and drag events is controverial and varies from window system to window system. That's why I didn't build them into the C substrate, but instead made them be processed up in lisp. (But this is a tangential point; if you want to talk more about it, send me mail.)

It also fixes the stupid problem where mouse commands leave a message behind only to have it erased when the release-event is processed.

Hey, this is just a *bug*, it has nothing to do with the event model per se...

B: All events have the same representation, which has a certain elegance to it. (This elegance is muffled by the need to keep the garbage collector away from event management, though.)

See above.

GNU Emacs simply has both. If there is a character in unread-command-char, it stuffs it into the input stream, and if there are events in unread-command-events, it stuffs those, too. There is some order of priority, but it doesn't really matter.

You could still have retained compatibility by using the name "unread-command-char" instead of "unread-command-event" for your current variable, and allowing it to contain integers as well as event objects.

The reason I didn't do this is that the canonical use of unread-command-char is something like this:

   (let ((c (read-char)))
    (if (= c 32)
	(progn ... )
      (setq unread-command-char c)))

If the user types some key with an unambiguous ASCII equivalent, this works fine. But what if the user types Control-Shift-A or Shift-Linefeed, which have no ASCII equivalent. Right now, lemacs's read-char signals an error when C-Sh-A is typed, and returns 10 for Sh-LFD (the same as LFD). This means that either this code would fail to work with many otherwise legitimate keystrokes, or would unread LFD instead of Sh-LFD, which could have a different binding. Likewise, when this code reads and unreads \010, we lose the distinction between Control-h and Backspace.

Again, maybe you think that compatibility is more important than the feature that BS and C-h aren't the same thing. That's a perfectly legitimate opinion. But I can't think of a way to have both.

Using the FSF19 model, an int would be returned for C-a and an "event" would be returned for C-Sh-A. But what about BS/C-h? Which one returns an event? Both are equally bad choices.

Maybe RMS' system has evolved since you first saw it. He represents function keys as symbols, and he can also represent a full set of modifier bits on printable keys. In terms of keys themselves, both systems are equally expressive as far as I can see.

The last time I checked, C-h, C-Shift-H, BS, C-BS, and C-Shift-BS were not distinguishable in the FSF version. Has this changed?

Yes, but there are about a zillion places in Calc where I stuff an event that I *know* is a printable character, and I had to change every single one of these.

I understand that you've had massive compatibility problems. But in my defense, I think that Calc is about as atypical an emacs application as you can find. Most packages just don't work that way; by far the most common use of unread-command-char is to unread an arbitrary character read from the user.

Your model does not detect incompatible code, either, because older programs can simply go ahead and store something in "unread-command-char" and then silently fail.

Yes, this is true. However, it does issue a compilation warning. I could make setq signal an error when storing into unread-command-char, but that seems pretty perverse...

-- Jamie

My first impressions of FSF v19 vs. Lucid Emacs 19: I liked having scroll bars and having horizontally-split windows, selective display etc. working, but the FSF menus are *tacky*. IMHO it's high time FSF emacs began to use XT and widgets properly -- anyone who has X has XT anyway. Lucid's event and keymap implementation is generally better.

In the Lisp library, however, FSF emacs was much more polished. Lucid Emacs's Lisp is (except for that which had to be changed due to changes in Lucid's C code) largely v18 with a hodgepodge of seperate packages tied on.

The GNU documentation was better, too.

I also liked the scrollbars, but the menus were horrid. And why did they choose to not go with motif or lucids motif style widgets for the menus and the scrollbars?

I would love the next version of LucidEmacs to have the capability to place vertical & horizontal scroll bars (motif style) on each edit area. Any plans for this?

Chuck

jwz@lucid.com (Jamie Zawinski) said:

The last time I checked, C-h, C-Shift-H, BS, C-BS, and C-Shift-BS were not distinguishable in the FSF version. Has this changed?

Yes.

The features provided by GNU Emacs 19.8 seem to be a superset of the features provided Lucid Emacs 19.4 (I have not tried 19.6).

Jamie Zawinski writes:

Keys and buffer-characters are not the same thing. Strings are made of the stuff that buffers are made of. Keys are not. Show me how to represent Control-Shift-A or Control-Backspace in a string.

But some keys are. In fact, statistically nearly all keys that are actually typed are. Likewise, many key sequences dealt with by programs are. It's silly not to incorporate this important fact into a program's concept of events.

Lucid Emacs does this anyway, by allowing strings in some contexts where event vectors are expected. But it's a kludge---only some contexts allow this, where others require you to use "event-to-character" and its inverse.

RMS' model represents all the key distinctions yours does, but the subset of keys which are ASCII characters are represented by their ASCII codes. It's very natural. All vector operations in Lisp apply to strings as well, which can be thought of simply as special vectors optimized to store only ASCII characters. So the relation between strings and event vectors also becomes very natural in RMS' scheme.

It's not that your system doesn't have its own advantages; it's just that I've seen some claims that your system is obviously and completely superior, which is simply not the case.

You don't need to resort to explicit memory management. You can drop the events on the floor and they will be garbage collected like any other lisp object. It's simply more efficient to reuse them [...]

I know, and that is commendable. It's purely a matter of efficiency, and as you say it only matters in one point in the command loop. But because of that one little thing, little uglies show up all over the place in Lemacs' event functions. I'm thinking of "next-event", which takes a blank event object to be filled in rather than simply returning a new event object. That's inelegant and un-Lispish; it's an expedient that isn't necessary in RMS' model.

It such a common misconception that you *need* to explicitly manage events that I'm starting to regret that I made it possible at all...

I never misunderstood; I just interpreted your design as deriving from a *need* to manage events in the command loop for performance reasons. If this kludge had been hidden inside the command loop it wouldn't have mattered, but in your design, "next-event" *must* take an event to be filled in rather than consing one up itself, "last-command-event" *must* be copied because it will be scribbled on by the next input, and so on. Again, your event model is hardly crippled by any of this, but it *is* inelegant in a system whose main virtue, as you've argued, is its elegance. RMS' model neatly sidesteps the whole issue. That, in its way, is elegant.

And harder to programmatically tell apart from other things [...]

Lists are used for all kinds of things right now in Lisp, and it doesn't seem to hurt anything. Lambda expressions can't be distinguished from lists except by context, but that's okay because context is enough. Likewise for events and keymaps. You know it's an event because you got it from a place that has events in it.

But perhaps they should be less opaque; give me examples where you've wanted this.

I'm tempted to, but I will resist---it misses the point. In some contexts, like large object-oriented software systems, opaque data types are a virtue. In others, flexibility and malleability are the prime virtues. I think Emacs falls in the latter class. For a system like that you don't want to wait for examples of why it would be good to have access to a particular thing, you want to give access to as much as possible *on the assumption* that some ingenious soul will later find a way to use it.

That's why I worry a little bit about the fact that RMS' model forgets the time stamps on keystroke events. I have only a vague sense of when these would be useful, but I have absolute faith that people, given this information, will be able to do cool things with it. (I had misremembered your event model as storing mouse positions in keystroke events, too, which could also be useful though less portable between window systems.)

I very carefully designed the event structures to expose only those components that emacs would need when running under *any* window system. They are not X specific at all; why do you say they are?

I went back and looked over your events, and, though they are X-like in character, I agree that they are no more X-dependent than GNU's.

The Lucid model acts on the click, whereas RMS's model usually acts on the release. Lucid delivers both up and down events, and you can bind commands to either.

Same in GNU Emacs 19, just that commands are now traditionally bound to up instead of down. But the way the default bindings works sets a pretty strong precedent for the way people will do their own bindings.

It also fixes the stupid problem where mouse commands leave a message behind only to have it erased when the release-event is processed.

Hey, this is just a *bug*, it has nothing to do with the event model per se...

Not with the event model itself but with the decision to process on up instead of down events. Emacs clears the echo area before processing a command; if the down click is an active operation then the up click acts like a null command with the side effect of clearing the last message. Fixing the bug with active down-clicks would require a kludge, but with active up-clicks the whole problem goes away.

You could still have retained compatibility by using the name "unread-command-char" [etc...]

The reason I didn't do this is that the canonical use of unread-command-char is something like this: [Reading a character and stuffing it back]

Again, maybe you think that compatibility is more important than the feature that BS and C-h aren't the same thing. That's a perfectly legitimate opinion. But I can't think of a way to have both.

I understand that the "read-char"/"unread-command-char" idiom now loses information. Nobody is asking you to forego an "unread-command-event" feature, just that providing a backward-compatible feature *as well* will allow code to hobble along until someone has time to fix it. You don't have to give one up for the other. Just have both, and then everybody is happy in both the short term and the long term.

The last time I checked, C-h, C-Shift-H, BS, C-BS, and C-Shift-BS were not distinguishable in the FSF version. Has this changed?

Yes. They are all distinguishable.

Using the FSF19 model, an int would be returned for C-a and an "event" would be returned for C-Sh-A. But what about BS/C-h? Which one returns an event? Both are equally bad choices.

The event for C-Sh-A is a number with a few high-end modifier bits set. The event for C-h is 8, and the event for BS is the symbol "backspace". This is rather like Lucid, where C-h is treated as a regular control character and BS is treated as a function key. You both had to make the choice, and you both happened to make the same choice.

RMS' model is more irregular in the way it handles modifier bits and things. Lucid Emacs is much more orthogonal there, which is nice. But I expect the two systems will be equally convenient to deal with in practice.

I could make setq signal an error when storing into unread-command-char, but that seems pretty perverse...

I'll second that! I thought of it, too, but I didn't want to suggest it in public. :-)

-- Dave

Will the next release of Lucid Emacs contain most of the Emacs Lisp changes to FSF v19.8? Obviously some things aren't compatible with the way Lucid Emacs works, but how about the rest?

My first impressions of FSF v19 vs. Lucid Emacs 19: I liked having scroll bars and having horizontally-split windows, selective display etc. working, but the FSF menus are *tacky*. IMHO it's high time FSF emacs began to use XT and widgets properly -- anyone who has X has XT anyway. Lucid's event and keymap implementation is generally better.

In the Lisp library, however, FSF emacs was much more polished. Lucid Emacs's Lisp is (except for that which had to be changed due to changes in Lucid's C code) largely v18 with a hodgepodge of seperate packages tied on.

The GNU documentation was better, too.

-Morven

And harder to programmatically tell apart from other things [...]

Lists are used for all kinds of things right now in Lisp, and it doesn't seem to hurt anything. Lambda expressions can't be distinguished from lists except by context, but that's okay because context is enough. Likewise for events and keymaps. You know it's an event because you got it from a place that has events in it.

I don't think you'll find much anybody involved in the Lisp world (apart from those with their tastes set in concrete circa 1972) who believe that the ad-hoc mess of "vectors of length 259 of lists whose first element is the symbol `indirect' and whose tail is an alist of conses whose ..." is any substitute for actual structure objects (of varous degrees of opacity.)

It is one thing to argue that such perversions might make sense for some reason of compatibility, but quite another to claim that representing hash-tables and keymaps and events and so forth in terms of conses and vectors and symbols is in any way a feature. (Why aren't windows and processes and window configurations represented in such a way in FSFmacs anyway?)

By the way, FSFmacs doesn't allow "(fillarray (make-keymap))" any more. This "incompatibly broke" a bunch of code I wrote.

Dave Gillespie wrote:

Lucid Emacs does this anyway, by allowing strings in some contexts where event vectors are expected. But it's a kludge---only some contexts allow this, where others require you to use "event-to-character" and its inverse.

I believe that all functions which take event vectors also take strings, and all functions which take events also accept ASCII integers.

unread-command-event must be an event, not an ASCII integer. I have no particular objection to making it able to take an integer as well, except that I don't see it as particularly useful (and the event- accessor functions would need to be modified to work on ASCII integers as well, or all code which references unread-command-event would have to be modified to check to see if it is really an integer, either of which I would consider an ugly wart.)

All functions/variables which return a key or sequence of keys return events or vectors of events. I think this is a good thing because of uniformity: keys are events, not sometimes events and sometimes integers. It makes less work for the programmer.

YES, it is incompatible. YES, it makes more work for people porting their code. I'm talking about purity of design, not about compatibility with existing code.

RMS' model represents all the key distinctions yours does, but the subset of keys which are ASCII characters are represented by their ASCII codes. It's very natural.

It makes more work for the programmer because the type of the key changes.

I know, and that is commendable. It's purely a matter of efficiency, and as you say it only matters in one point in the command loop. But because of that one little thing, little uglies show up all over the place in Lemacs' event functions. I'm thinking of "next-event", which takes a blank event object to be filled in rather than simply returning a new event object. That's inelegant and un-Lispish; it's an expedient that isn't necessary in RMS' model.

You're right; the arg to next-event should be optional.

Lists are used for all kinds of things right now in Lisp, and it doesn't seem to hurt anything.

(Mly's reply said exactly what I would have said to this.)

For a system like that you don't want to wait for examples of why it would be good to have access to a particular thing, you want to give access to as much as possible *on the assumption* that some ingenious soul will later find a way to use it.

This is far from clear to me. The fact that "everybody knows keymaps are just alists/vectors" was a big problem. Lots of code got written that used aset and fillarray instead of the "advertised" interface of define-key, making it harder to add power to keymaps (in both lemacs and FSFmacs.)

It also fixes the stupid problem where mouse commands leave a message behind only to have it erased when the release-event is processed.

Hey, this is just a *bug*, it has nothing to do with the event model per se...

Not with the event model itself but with the decision to process on up instead of down events. Emacs clears the echo area before processing a command; if the down click is an active operation then the up click acts like a null command with the side effect of clearing the last message. Fixing the bug with active down-clicks would require a kludge, but with active up-clicks the whole problem goes away.

Well I really think we're getting off the track here, but it is my opinion that if one *does* choose to bind a command to a downstroke (whether that's the prescedent or not), and there is no command on the upstroke, then the upstroke should not cause the minibuffer to be cleared. This used to work in lemacs, but it's broken right now.

I understand that the "read-char"/"unread-command-char" idiom now loses information. Nobody is asking you to forego an "unread-command-event" feature, just that providing a backward-compatible feature *as well* will allow code to hobble along until someone has time to fix it. You don't have to give one up for the other. Just have both, and then everybody is happy in both the short term and the long term.

I think that having existing code hobble along in way that sometimes loses information is not a good solution. I have never subscribed to the Unixlike idea that "mostly working most of the time" is good enough. I think it's better for incompatible changes to cause code to not work at all than to lose in obscure ways. Otherwise it will never get fixed.

If I could see a way to cause existing code to work 100% without compromising the design, I would do it. If I thought that compatibility with v18 was important enough to compromise the design, I would do it.

The event for C-Sh-A is a number with a few high-end modifier bits set. The event for C-h is 8, and the event for BS is the symbol "backspace".

Then in FSF19, there are really *three* representations of events: ASCII codes; ASCII-like codes with bits beyond #7 set; and symbols. In lemacs, there is but a single representation (with strings allowed as well in the interest of compatibility). The FSF model is good for compatibility, but I think my model is a cleaner (and "more Lispy") way of doing things.

-- Jamie

mjb@doc.ic.ac.uk (Matthew J Brown) writes:

In the Lisp library, however, FSF emacs was much more polished. Lucid Emacs's Lisp is (except for that which had to be changed due to changes in Lucid's C code) largely v18 with a hodgepodge of seperate packages tied on.

For example?

The more important questions however is will Lucid and FSF Emacs keep jumping each other's minor version number? Lucid's next release should be Lucid Emacs 19.27 just to see if the FSF jumps to 19.>28.

- Ed

Richard Mlynarik writes:

It is one thing to argue that such perversions might make sense for some reason of compatibility, but quite another to claim that representing hash-tables and keymaps and events and so forth in terms of conses and vectors and symbols is in any way a feature.

GNU Emacs doesn't have hash-tables of any sort, as far as I can tell. If it did you're right that they would work better as primitive Lisp objects. It remains to be seen whether GNU Emacs 19 will suffer from using alists instead of hash tables to store function key and mouse maps. If the maps get big enough, it will.

The GNU keymaps are admittedly pretty baroque.

(Why aren't windows and processes and window configurations represented in such a way in FSFmacs anyway?)

Windows and processes probably contain data that is more conveniently dealt with on strictly a C level. There might also be funny business in there with the garbage collector; there certainly is with buffers and markers.

I've often wondered why window configurations are opaque. It has occasionally been frustrating that they were.

By the way, FSFmacs doesn't allow "(fillarray (make-keymap))" any more. This "incompatibly broke" a bunch of code I wrote.

Very true. I see why they had to do it, but it's still a shame that keymaps couldn't be 100% backward compatible.

-- Dave

Jamie Zawinski writes:

I believe that all functions which take event vectors also take strings, and all functions which take events also accept ASCII integers.

I just did "(setq unread-command-event ?a)" and it gave an error.

unread-command-event must be an event, not an ASCII integer. I have no particular objection to making it able to take an integer as well, except that I don't see it as particularly useful [...]

Nor do I, on its own terms. My suggestion was that, if the variable allowed this and still retained the name "unread-command-char", then existing Lisp code would be able to function more or less. Not perfectly, but it would work until the various authors had time to redo it properly.

I'm talking about purity of design, not about compatibility with existing code.

And I appreciate the purity of your design. It would be kludge for Lucid Emacs to return strings from "this-command-keys" sometimes. But it's not a kludge in GNU Emacs, because a vector of ASCII events in GNU Emacs would be a vector of byte-sized integers, for which strings can serve as functional substitutes.

There is a kludge for meta characters in the GNU system, though. I can see it possibly coming back to haunt them some day.

It makes more work for the programmer because the type of the key changes.

Could be, although I think it's rarely going to matter much. GNU Emacs 19 does provide a few accessors for getting at things like the modifiers of any key.

Lots of code got written that used aset and fillarray instead of the "advertised" interface of define-key, making it harder to add power to keymaps (in both lemacs and FSFmacs.)

True, and I've certainly been guilty of that myself.

Honestly, I think the balance of merits is a lot more interesting on the issue of events than on keymaps. Lucid's keymap model works perfectly well, and even I, as devil's advocate, only managed to come up with half-hearted arguments in favor of vectors and conses. But RMS' event model has a nice feel to it. It still could be nice to add extra stuff to events in the future, which his model can do for mouse events but not for keystrokes. But there are other advantages that weigh against it.

I think it's better for incompatible changes to cause code to not work at all than to lose in obscure ways. Otherwise it will never get fixed.

-- Dave

It is one thing to argue that such perversions might make sense for some reason of compatibility, but quite another to claim that representing hash-tables and keymaps and events and so forth in terms of conses and vectors and symbols is in any way a feature.

GNU Emacs doesn't have hash-tables of any sort, as far as I can tell.

"intern".

(lemacs has more general (non-string-keyed) tables.)

Dave Gillespie wrote:

I believe that all functions which take event vectors also take strings, and all functions which take events also accept ASCII integers.

I just did "(setq unread-command-event ?a)" and it gave an error.

That's a variable. (And I explained in my last message why I didn't make it able to take integers: because, being a variable, it "returns" a value as well as taking an "argument", and I don't want integers to be "returned" for events.)

Nor do I, on its own terms. My suggestion was that, if the variable allowed this and still retained the name "unread-command-char", then existing Lisp code would be able to function more or less. Not perfectly, but it would work until the various authors had time to redo it properly.

I understand. And my argument against this was the such code would only work most of the time, and the existing bug would be obscured.

It makes more work for the programmer because the type of the key changes.

Could be, although I think it's rarely going to matter much.

It may well be that the existing code will only fail in rare cases, that most users don't encounter. But I don't think that's a good thing. For code which uses read-char and unread-command-whatever to be *correct* in the FSF model, it must accept that events can be integers or symbols. Existing code which assumes they are integers will work if all you type is keys which are represented integrally, and those are the majority. But the code is still incorrect, and will lead to obscure errors that are unlikely to be noticed for a long time.

; I think it's better for incompatible changes to cause code to not work at all than to lose in obscure ways. Otherwise it will never get fixed.

Then perhaps Lucid's command loop should yell if it sees that unread-command-char has been bound to anything.

(I guess you mean "set", not "bound.") Yes, I think that's a good idea.

-- Jamie

GNU Emacs doesn't have hash-tables of any sort, as far as I can tell.

"intern".

(lemacs has more general (non-string-keyed) tables.)

Sigh. :-] Well, I was referring to general-purpose hash tables, with any sort of implementation, in GNU's version of Emacs 19. Not that it really matters...

-- Dave

Dave Gillespie writes:

[...] I worry a little bit about the fact that RMS' model forgets the time stamps on keystroke events. I have only a vague sense of when these would be useful,

How about double-clicking on some keys (the space bar, for instance)? If it is useful with the mouse, I can see it being useful on a mouseless terminal

I believe that all functions which take event vectors also take strings [...]

I just did "(setq unread-command-event ?a)" and it gave an error.

That's a variable.

... which was my point: Unless the symmetry between events and characters is carried all the way through, there will always be something that slips through the cracks. Even if the functions can all be fixed, there will be variables that break and can't easily be fixed (for the reasons you stated).

Then perhaps Lucid's command loop should yell if it sees that unread-command-char has been bound to anything.

(I guess you mean "set", not "bound.") Yes, I think that's a good idea.

I was thinking "boundp", but yes, I suppose "set" is the right word.

-- Dave

Dave Gillespie wrote:

... which was my point: Unless the symmetry between events and characters is carried all the way through, there will always be something that slips through the cracks. Even if the functions can all be fixed, there will be variables that break and can't easily be fixed (for the reasons you stated).

Rather than seeing this as an omission of the design, which "slips through the cracks" implies, I see this as a piece of backward compatibility which is simply not provided. Lemacs uses strings and ASCII codes only for compatibility; if I could get away with it without breaking every .emacs file that has ever existed, lemacs wouldn't use them at all. The backward compatibility is there in places where it wouldn't adversely affect the core of the design, which I think is clean and consistent. The compatibility kludges are not complete, because I don't think they are worth the price.

-- Jamie

On Wed, 26 May 1993 14:55:22 PDT, mly@adoc.xerox.com (Richard Mlynarik) said:

I don't think you'll find much anybody involved in the Lisp world (apart from those with their tastes set in concrete circa 1972) who believe that the ad-hoc mess of "vectors of length 259 of lists whose first element is the symbol `indirect' and whose tail is an alist of conses whose ..." is any substitute for actual structure objects (of varous degrees of opacity.)

No, they went ahead and implemented monster lisp environments that collapsed under their own weight during the 80'ties. The techniques needed to survive in those environments are not the same as for a lean and mean lisp environment such as GNU Emacs.

Apart from that, I would agree that Jamie Zawinski's event model is better. It is just that the benefit of having a `clean' event model is much smaller than the problems of having two different event models. Keymaps are not a problem, since GNU emacs can emulate Lucid's interface.

Ed Gamble wrote:

The more important questions however is will Lucid and FSF Emacs keep jumping each other's minor version number? Lucid's next release should be Lucid Emacs 19.27 just to see if the FSF jumps to 19.>28.

Jimb and I don't really have any good ideas what to do about this; he says that he didn't pick 19.7 because the last lemacs was 19.6, but that by some great, cosmic coincidence, there really were 6 alpha releases of FSF's version...

I'll probably call the next lemacs 19.7, and just hope that people mention which one they mean. `emacs-version' is "19.6 Lucid", after all.

-- Jamie

I don't think you'll find much anybody involved in the Lisp world (apart from those with their tastes set in concrete circa 1972) who believe that the ad-hoc mess of gt; "vectors of length 259 of lists whose first element is the symbol `indirect' and whose tail is an alist of conses whose ..." is any substitute for actual structure objects (of varous degrees of opacity.)

No, they went ahead and implemented monster lisp environments that collapsed under their own weight during the 80'ties. The techniques needed to survive in those environments are not the same as for a lean and mean lisp environment such as GNU Emacs.

How stupid of me to forget.

PS I hope you never use "struct" in your C code.

I personally find it very difficult to argue in favor of the GNU keymap structure. It seems to me that building keymaps using the simple types has the following arguments in its favor:

1. It requires no changes to allocation code or GC code - things do break.
2. It requires no new read syntax or printed representation.
3. You don't need to worry about thinking of all the necessary primitives beforehand, since people can roll their own.
4. It's obvious, just by looking at the structure, how they work.
5. You get leverage from the ability to apply existing routines (assq, etcetera) to them.

But those are kind of stretched.

1. Is is really that hard to write allocation, parsing, and unparsing code? ("Are we not hackers?" :-) )
2. Come on, it's not hard to come up with a good set of primitives. A keymap is a function mapping events onto commands. How many things do you need to do with that?
3. It's *not* obvious how to use them any more! Did you know that a binding for t in an GNU keymap structure acts like a default, catch-all binding, providing the only way to create a keymap that catches *all* events? And exactly what does it mean to nconc one keymap onto another? Are you *sure*?
4. You can no longer effectively apply existing utility routines like assq to an GNU keymap. assq doesn't know about the vectors. The keymaps are too hairy to be operated on by anything but code which knows it's dealing with a keymap.
5. It's a pain to upgrade. The keymap changes are probably the largest source of broken a lot of code in GNU 19. Anyone who thinks it's not going to happen again is a doodyhead.
6. It restricts your implementation. If you expect people to poke around in the data structure in the raw, this discourages you from making reasonable improvements, like the Lucid reverse indices.

I have yet to hear a single mildly persuasive argument from anyone in favor of the GNU keymap structure.

RMS is this great programmer. He's got GCC. He's got Emacs. He's got the MacArthur. He's got this luxurious pad on the fourth floor of posh NE43. I go to work for him, hoping some of that will rub off. It turns out that the single most common theme in his thought is, "That's too much work." "That's going to take too long." "Don't bother."

Is that what it takes? Is this the punchline of my lifelong pursuit of hackerdom? I'm apprenticing myself to a blacksmith, just in case. :-)

On Wed, 26 May 1993 16:14:18 GMT, hines@brooklyn.fishkill.ibm.com (Charles Hines) said:

My first impressions of FSF v19 vs. Lucid Emacs 19: I liked having scroll bars and having horizontally-split windows, selective display etc. working, but the FSF menus are *tacky*. IMHO it's high time FSF emacs began to use XT and widgets properly -- anyone who has X has XT anyway. Lucid's event and keymap implementation is generally better.

In the Lisp library, however, FSF emacs was much more polished. Lucid Emacs's Lisp is (except for that which had to be changed due to changes in Lucid's C code) largely v18 with a hodgepodge of seperate packages tied on.

The GNU documentation was better, too.

I also liked the scrollbars, but the menus were horrid. And why did they choose to not go with motif or lucids motif style widgets for the menus and the scrollbars?

Motif cost money! Is the lucid widget stuff free? What are the licensing restrictions on this?

; I would love the next version of LucidEmacs to have the capability to place vertical & horizontal scroll bars (motif style) on each edit area. Any plans for this?

That would be a very welcome addition, that would help convert a lot of anti-emacs types around here.

Hmm, maybe: Lucid Emacs 20.1 would solve the problem ;-)

In article <EBG.93May27074629@hoshi.hip.atr.co.jp> ebg@hoshi.hip.atr.co.jp (Ed Gamble) writes:

The more important questions however is will Lucid and FSF Emacs keep jumping each other's minor version number? Lucid's next release should be Lucid Emacs 19.27 just to see if the FSF jumps to 19.>28.

At the rate that GNU bug fix versions are comming out, maybe 27 is too small a nnumber for Lucid to avoid confusion with a soon to exist GNU version.

-jason

The release of GNU Emacs 19 from the FSF does not mean that Lucid Emacs is going away. We will continue developing it, and merging in bug fixes and new features from the FSF version as appropriate; we do not have plans to discard the functionality that RMS has chosen not to include in his version.

The long awaited Great Redisplay Rewrite is well underway, in the new guise of the Great Epoch Merge. Most of this work will end up being done by the Epoch developers, but we're in close communication.

Within two months we expect to have a version of Lucid Emacs with the new redisplay, which can run all Epoch 4 code without alteration (excepting those areas where lemacs is incompatible with v18, that is.)

Lucid Emacs will run on dumb ttys shortly thereafter (if not at the same time.)

I will probably release a new version of lemacs in a few weeks which contains some bug fixes and portability improvements, and a bunch of internal improvements that aren't user visible. The next release after that will probably be in a few months, when the Epoch merge has reached a point of stability.

The latest released Lucid Emacs is 19.6, and can be found on labrea.Stanford.EDU in pub/gnu/lucid/; see the README.

There is a version of Lucid Emacs in beta test now; send me mail if you'd like to help out with it (hackers only, please.)

-- Jamie

Neal Becker wrote:

I also liked the scrollbars, but the menus were horrid. And why did they choose to not go with motif or lucids motif style widgets for the menus and the scrollbars?

Motif cost money! Is the lucid widget stuff free? What are the licensing restrictions on this?

Lucid Emacs can be compiled with Motif if you have it, but also comes with a simple set of Motif-lookalike widgets used for the menubar and popup menus. This code is Copylefted; all you need is Xt. (Even if you have Motif, I recommend that you use the Lucid widgets, since they're loads faster (though not 100% compatible).)

I would love the next version of LucidEmacs to have the capability to place vertical & horizontal scroll bars (motif style) on each edit area. Any plans for this?

Yes. Not in the next release, but soon.

-- Jamie

Matthew J Brown wrote:

Will the next release of Lucid Emacs contain most of the Emacs Lisp changes to FSF v19.8? Obviously some things aren't compatible with the way Lucid Emacs works, but how about the rest?

Probably not, because I don't have time. The current beta version of lemacs has had many FSF changes merged in to it, along with a lot of internal improvements. Most of the files in the lemacs lisp/prim/ directory have been merged against their FSF counterparts, but the rest of the lisp library has not.

This is something that a volunteer could easily do, since it doesn't require intimate knowledge of lemacs internals...

The GNU documentation was better, too.

Likewise, I don't have much time to hack documentation if you want all of these redisplay bugs fixed...

I'm also still looking for some noble volunteer to add "configure" support to lemacs. This should be quite simple with the current lemacs beta, since at this point, the lemacs s/ and m/ directories are basically identical to their GNU19 counterparts.

-- Jamie

I recall a message in February where you posted a list of the features you would find essential in order to be able to use Emacs 19. You asked for the FSF to show the will to merge the two versions of Emacs by doing some of the work.

I did this. Emacs 19 contains several features merged from Lucid Emacs, such as the X selection code and faces. It also contains comparable menu and event features with emulation for the Lucid interfaces. It contains features comparable with intervals, and emulation for the Lucid interval functions should be easy to add. Almost all of the features on that list are now provided.

So it would seem proper for you to pitch in and merge things such as toolkit output, sound output, and dialogue boxes into Emacs 19 and end the split.

Your posting today seems to describe quite different plans, however. Are you willing to end the split, or not?

We have been doing a lot of merging with your sources, but in the direction of taking your new features and putting them in our source tree. As this results in the two emacses being closer together, I don't think our current strategy is incompatible with an eventual merge with your version of emacs; it will help in the long run.

We need to talk about this more on our end (some people who should be involved are on vacation for a few days) and will send you mail later.

-- Jamie

Please don't use motif-style scrollbars. Remember, this is emacs, so I'd like to be able to control what mouse buttons are bound to in the scrollbars. I don't care if they look like motif scrollbars, just that they don't have to act like them.

I hate motif (and Open Look, for that matter) scrollbars because I'd like one button to scroll up and another to scroll down, and another to scroll absolutely ala xterm scrollbars. It annoys me that I have to click before the gizmo or after the gizmo. It really anoys me when the gizmo moves up past my mouse cursor and the same clicking which was scrolling up now starts to scroll down.

I'll bet that once you add scrollbars, people will want to bind their own functions to do interesting things.

Hayden

I just want to say to keep up the good work. I've just tried emacs 19.12, and I must say that lemacs 19.6 is quite superior! Especially the menubars. I am hopefull that 19.7 will have scroll bars. I also like the way the middle mouse button is used in completions, etc. Thanks to Jamie et. al.

I'll second (and third) the kudos. Since I brought up Lucid emacs, I haven't even bothered to bring up GNU 19.<todays release>, and the 18.xx version that I left lying around gets used when I dial in [no PPP/X yet :-( ]

Congrats, keep up the good work, and "Thanks!",

Steve

Have you decided whether you'd like to work with me to merge the two versions?

I have been wanting to ask for volunteers to merge the Lucid Xt interface code into Emacs 19. I have put it off because, if you want to merge the versions, that is better than finding someone else. But it is getting to the point where I ought to ask for someone else if you don't want to do it.

Tomorrow at the GNU Bof, it will behoove me to ask for volunteers to work on things like merging the Xt support into GNU Emacs.

I was hoping to announce that you were going to do this. Now I guess you aren't going to, but I must carry on.

Richard:

We got together and discussed the idea of completing the merge of our code into your version decided against it. The primary reason is that the structure of GNU Emacs 19 and its lack of clean internal interfaces are fundamentally unmaintainable, and we believe that trying to maintain it along with the additions to Emacs we need and adding the things we need would be too expensive for us.

Unfortunately, GNU Emacs 19 is not constructed following modern programming practice - even though there are procedural interfaces to some functionality groups, one could not say that abstraction boundaries are maintained. The mish-mash of data structures is just too clumsy and incoherent to make life for the maintainer easy. This is proven by the fact that Arceneaux was unable to add the extensions we needed to an early version of GNU Emacs 19 given somewhat over a year's time to do it and by the fact that it has taken you and Blandy so long to get GNU Emacs 19 out.

We believe that the merge in the other direction is the proper way to do things, and this is the course we shall pursue. There are things that you have done to GNU Emacs 19 that are good, and we believe that the best way to proceed is to clean them up and put them in Lucid Emacs 19.

Maintainability is important to us because even though GNU Emacs 19 has no warranty, we must support a warranty on our commercial product which includes Lemacs as a component.

I think this is not a good outcome for the Emacs user community, and I hate to have to criticize the implementation of GNU Emacs 19, but at some point clean code and good structure counts more than a false sense of parsimony, and that point has been reached.

-rpg-

I am sad you won't help.

I looked at the Lucid Intervals code, intending to merge it in, but changed my mind because I couldn't understand it.