Writing a Technical Book
IronPython in Action with Manning Publications
|Date:||July 13th 2009|
|Book:||IronPython in Action|
Stating the obvious
IronPython in Action is a book on the Microsoft open source implementation of Python for Mono and the .NET framework. It is written by Michael Foord and Christian Muirhead. It covers application development, scripting and embedding IronPython in .NET applications - with chapters on WPF, Silverlight / Moonlight development and a Python tutorial. Christian and I are both experienced Python programmers who have been developing full time with IronPython for Resolver Systems since about IronPython version 0.7.
Any day now I will get the first quarter sales figures for IronPython in Action. That will mark the book having been actually in the hands of readers for three months and also be about two and a half years since I first contacted Manning about the possibility of writing an IronPython book for them.
As I always suspected, in retrospect I greatly enjoyed the process and challenge of writing a book. It has however left me with a strong suspicion that the three laws of optimizing code can be neatly modified for technical book writing:
- No really: don't
- If you absolutely have to, go read the contract section at the end of this article
I started working with IronPython in 2006 when I joined Resolver Systems. As IronPython was still very new (version 0.6 or 0.7) there were very few people within the Python community using it seriously so I started to write a tutorial series. I enjoy writing, and there were no books on IronPython so I decided to see if a publisher would be interested. I have a contact who works with O'Reilly and got in touch with them. Having reviewed (and been impressed by) a few Manning books I also contacted them. After initial discussion O'Reilly never came back to me - so it gave me some satisfaction to later hear of them hawking around the IronPython mailing list trying to find someone to write a book for them.
My original idea was to write a book on IronPython and Windows Forms as that was what I was spending most of time working on; but Manning wanted a much more general book on developing with IronPython that would fit into their "in Action" series. I first exchanged emails with Mike Stephens in June 2006. Over a period of six months I worked through turning a vague idea into a full book proposal. Writing a Manning proposal is a lot of work, including writing a complete table of contents down to the sections and sub-sections.
Early on whilst writing the proposal I roped my colleague Christian Muirhead in. I was to write thirteen of the fifteen chapters whilst he was to handle the two chapters on ASP.NET and databases and webservices.
We finally started writing in the backend of December 2006. The book was complete, including index, front matter and all the edits, in January 2009. It was in the shops for people to buy in early April 2009. That's a long time.
Perhaps the main reason it took so long is that I was working full time throughout, with a 2 hour commute in each direction. If I was lucky and got a seat on the train then about an hour of that I could work on the book but getting a 3G modem with internet connection put paid to using the time productively... (Eventually my pattern became that I would work late into the night and then use the time on the train to catch up on my sleep.)
Because of working for Resolver Systems the time I had for the book was after returning home and weekends. Often in the week I would get home after 8pm and between eating, dealing with essential email (and essential RSS feeds) and being exhausted I would get maybe a couple of hours an evening to work on the book. Inevitably my most productive times were the weekends. IronPython in Action swallowed most of my weekends for the better part of two years.
There were various reasons why it took so long. Some of these are down to me, some unavoidable due to unforeseen events like the release of IronPython 2 and Silverlight, and some due to the Manning processes. Writing a book, particularly one on a hot new technology, has an inevitable tension between wanting to get the book to market as quickly (and therefore as cheaply - man hours on the editing cost bucks) and producing as high quality a book as possible. Some publishers manage to get books out very quickly, and often it shows. In the brave new online world perhaps this is the right way to go - it used to be possible to earn a living from writing technical books alone, but very few people do that now.
In March 2009 I was at PyCon 2009. Although the book was complete it was not quite out yet, we missed PyCon by a couple of weeks. I mentioned to Guido (yes I'm a fanboy) that I'd finished writing the IronPython book. On hearing that it was for Manning he made the telling comment that they had a reputation for taking a long time to actually get books published. Although things didn't go perfectly with Manning I think we could only realistically have completed the book six months faster. This would have got us in time for the Christmas 2008 market, so it's a real shame that we didn't, but even so I am convinced that the tortuous process Manning puts its authors through does result in a higher quality book. I'm proud of what we've achieved with IronPython in Action - it is a fantastic book, and the we includes the whole team of folk from Manning who were involved. As Christian and I wrote in the acknowledgements:
"Writing this book has been a labor of love for the past two years. One thing that has astonished us is the sheer number of people involved in such an endeavor and how many individuals have helped us."
What follows is a look at the process I went through, including some necessarily negative comments. Please bear in mind that my overall impression is still positive and that much of what I say was specific to IronPython in Action and not what every author for Manning will experience.
After the proposal comes the writing. With Manning the writing is divided into various stages. From memory  it goes something like:
- Write a chapter
- Editor reviews the chapter and makes comments - the author responds to the comments making changes where necessary
- When a third of the book is complete the manuscript so far goes out to review; by a bunch of folk that Manning have found and a bunch of people who the author finds
- As their reviews come in the author makes any changes necessary in response
- When the whole book is complete it goes through a technical review from an appropriate expert
- The book then goes through copy-editing and typesetting
- Completing front matter and indexing
Technical review, copy-editing and proof-reading all involve changes that the author(s) must check and respond to.
All changes/questions/comments from the editor, technical reviewer and copy-editor are done using the Word change tracking system. Change tracking is where Word shines. All the changes must be individually accepted or rejected by the author and the comments responded to. This can be arduous but it does leave the author in control - all changes are just suggestions and it is up to the author which to accept and which to reject.
During the early stages every completed chapter is made available to readers (who pay) through the early access program. Feedback from the author's forum during this time is also folded back in to the book. The manuscript is managed through Manning's web application called Gregor which is a big document management system. Not too painful to use but Gregor is written in Java and it shows...
The first bump in the road was the Manning template system. Whilst I was writing IronPython in Action the manuscript was to be done as Word documents using a custom template for styles and one document per chapter. I did try opening some chapters in Open Office but it wasn't a pretty sight I'm afraid. The Manning style guide is a small book in itself and after completing the first draft of chapter 1 my editor of the time (I switched editors around chapter 3) pointed out a style rule I hadn't applied. After applying the rule he then came back and pointed out another one. This went round in circles for a while until he was finally ready to review the chapter. It took a month from me first completing the draft to it actually getting reviewed.
The second bump was a big one. Before the first third of the book (part 1) was complete I had a conversation with Marjan Bace, the Manning head honcho, which really threw a spanner in the works. The first part of the book, or at least the chapters after the introductions to Python and .NET, were focussed around the example Windows Forms application that I had originally envisioned being the backbone of the book. He worried that the book was too example focussed and needed to be driven by the principles rather than the code. Of course he was right, and going back and rewriting / reorganising three chapters greatly improved the final result. It cost nearly two months of time and it would have been much nicer to have fixed this during the proposal phase rather than after having written a huge chunk of the book.
Another bump in the road was the Word templates that Manning use for their styles. First of all Word uses fixed locations for the templates, meaning that opening a chapter on a new machine would break the template. Even worse, at some stage during the writing I switched first my laptop and then my home desktop from a Windows box to a Mac. Mac Office 2008 is actually a very capable application which I prefer to the Windows version of Microsoft Office. Although Manning has a template for Mac Office it was for a version prior to 2008 - and Office 2008 lacks VBA scripting support. The Manning template has custom support for indexing (etc) implemented in VB. Whether it is a bug in Office 2008 or a side-effect of the Manning template I found that actually doing the indexing on the Mac would crash every few entries. Infuriatingly frustrating and a massive productivity killer.
Another delay was entirely my own fault. After every third of the book it goes to external reviewers. Although mutually contradictory they came up with a lot of useful comments on the book as it progressed. Each time it would take about a month from the review copies going out to all of the reviews coming in. Instead of using that time to plough ahead on the next chapters I mainly used it to catch up on my blogging... My bad. These delays probably cost us two months.
IronPython 2 and Silverlight. Halfway through the book Microsoft announced IronPython 2, the Dynamic Language Runtime and Silverlight. I had to go back and reorganise - losing one of the introductory chapters and adding a chapter on Silverlight. Chunks of the introduction needed rewriting and lots of other places updating. Worse, IronPython 2 and the Silverlight version with DLR support were only in alpha and whilst writing I had to track changes from Silverlight 1.1 through to Silverlight 2 RTW and from IronPython 2 alpha to IronPython 2 final. API changes, particularly in the embedding API meant frequent changes to examples. Thankfully in the Silverlight chapter the changes usually meant removing ugly workarounds for bugs in the pre-release versions. On the other hand if I'd been much quicker with the getting the book out it would have out of date within days...
The book was basically finished around August 2008. All chapters were complete and it had been through the final third review. At this time my editor was telling me that we were on track for being in the shops in November. We started working through the myriad next stages, albeit slowly. I kept asking if we were still on track for a November release until in October, despite the assurances of my editor, it became obvious it just wasn't going to happen. It turned out that Manning had a bit of a schedule collision and didn't have the capacity in copy-editing / layout to get the book the attention it needed. This delay cost us another three months.
The final delay was minor but annoying. The Manning instructions for producing print ready images are fairly lengthy and I just couldn't get Open Office Draw to export the diagrams correctly in an acceptable format. As I'd specifically stated up front that I wouldn't have anything to do with typesetting and layout I finally gave up and insisted that they produce print ready images. Manning found someone in their typesetting department who was willing to install Open Office and things moved more swiftly.
There are some positives to writing a book:
- The money. Our advance was $5000 split 80/20 between Christian and myself. If the book pays off the advance we will make more on royalties. Based on the amount of time we put into the book this works out at something slightly below slave labour rates. Never write a technical book for the money. No really. Spend your evenings in McDonalds if you need extra cash.
- A different set of people will read the material than if I had put it up for free on the web.
- It was a challenge. I've never written a single work of this length before and I like a challenge. Without the pressure and commitment of a contract I would never have produced the comprehensive guide to developing with IronPython that IronPython in Action became. Taking on and completing a task of this size is very satisfying.
- I learned an enormous amount, particularly through the editing process. My writing has improved a great deal.
- I enjoy IronPython and I think Python is a fantastic programming language. I want Python to be more accessible and I enjoy sharing things I know. Writing a book on IronPython seemed to be a good way of achieving these things whilst putting something back into the communities that have given me so much.
- It looks bloody good on your CV and brings recognition as being an expert in the field...
If points three, four and five (suitably modified of course) on this list aren't your main reason for wanting to write a book then I wouldn't bother.
There are several negatives of course:
- The money is terrible.
- Less people will read my work than if I had put it on the web. Some of my articles are read by tens of thousands of people in a year. Even the less popular ones are read by thousands. I'll consider myself lucky if the book is read by ten thousand people in total. The counterbalance to this is that people will buy and read the book who would never read it if it was on the web. Those who do read it will probably read at least a significant chunk of it whereas if it was on the web many people would only read individual pages or perhaps chapters.
- It is a considerable amount of effort. A lot. I entered it with no preconceived idea of how long it would take and a determination to complete. This is a mindset I can recommend.
The ideal (at least to my mind) is to have the book available for free on the web under a decent free license (preferably Creative Commons) as well as being published. There are plenty of precedents for this; Dive into Python, A Byte of Python, The Django Book, anything by Cory Doctorow to name a few. If I do write another book (and despite my misgivings I have a couple of really good ideas) I will probably pre-empt this by making as much of the book as possible available online before approaching a publisher and make keeping it free a contract term. I don't believe this would hurt sales. Quite the opposite: obscurity is a greater enemy than giving away your stuff.
Here are some of the things I learned or should have learned. Most of these I guess I will have to relearn if I ever put myself through it again:
The most important thing I learned was don't sweat the small stuff. This warrants repeating. Don't sweat the small stuff. Many times I knew the gist of what I wanted to say in a passage but couldn't find the words. I would go round and round over a single sentence for fifteen minutes or more. This happened a lot. I learned to just write something and then come back to it later. Often what I had been unhappy about when writing read fine when I came back to it the next day. If I was really stuck I would just leave a placeholder (like XXXX or something easy to search for) and come back to it another time. Letting yourself get stuck drags out the writing process and makes it mentally exhausting. Far better to just write what you have and move on; you're going to be going back over it later anyway.
I can learn anything. This is a lesson I've had to learn many times throughout my programming . For several of the chapters or sections, for example the Silverlight and WPF chapters, the subject matter was completely new to me (when Silverlight came out it was new to everyone of course). That feeling of being out of your depth when you start a new topic is now familiar to me and I don't let it worry me; given time I can learn anything. Some of the subjects I had to learn I had the familiar completely-out-of-my-depth-and-don't-know-where-to-start feeling. This was particularly true when trying to negotiate the relationship between XPS and the FlowDocument classes. As usual a combination of research and just-giving-up-and-writing-some-damn-code won through. It is this feeling, and in particular this time spent, that technical books can circumvent for their readers. We do the research so that you don't have to.
There will always be mistakes, both typos and technical errors. The book went through umpteen stages of review, editing, corrections and changes. Of course whilst every change may correct a mistake it is also an opportunity to introduce new mistakes. So far the errata I have collected have been very minor, thanks largely to the number of review stages and some helpful folk who assiduously reported every typo and ambiguity they found during through the early access program.
Get a co-writer, or two, if you possibly can. Even though Christian only wrote two of the chapters and one of the appendices it made a huge difference having someone to share the burden. Having someone write half the book would have been much more sensible.
A good technical editor makes a huge difference. My gratitude and thanks to Jeff Bleiel for his input, questions and suggestions. The editor is there to help you - to explore ideas and possibilities. It is perfectly acceptable to say no to those suggestions and it is important to keep your own voice. I had one big conflict with Manning which I eventually gave in on - my only big regret. Although the book has two authors the parts written by me and the parts written by Christian are clearly delimited. My natural style is to use 'we' to refer to me and the reader, and to use I when speaking of myself. For example I would talk about how we are about to investigate a new subject. For books with more than one author Manning prefer to stick to 'you' for the reader and 'we' for the authors. This is less conversational and also a fiction as no single part of the book was written by a 'we'... Consistency is important, and it isn't a big deal, but it would have been more true to my own voice to have insisted on my style.
Presentation is important. The copy-editing (and applying the dreaded Manning styles consistently to the manuscript) made a massive difference. What Christian and I had produced actually looked like a professional book after copy-editing.
Be principle driven rather than example driven. Focus on the principles that code illuminates and not the code itself. This applies to the structure of the chapters as well as the content.
You don't need to include the full code of every example you work through, particularly if you are incrementally building something. Instead make the full examples available for download and explain that the code shown is incomplete. Some readers prefer every code example to be fully executable on its own, but for any non-trivial example showing it in full will double the amount of code shown. It also repeats the same boilerplate and buries the important part of the code. Some people will complain, but it is still better to only include in the book the code you need to illustrate the points you are explaining.
Create your examples using a test driven approach or be extremely disciplined about testing. When you refactor or change your examples having comprehensive automated tests makes it less painful. I didn't do this and regretted it several times. The ideal is to keep your examples in separate files and test directly from them. This doesn't play well with using Word documents for the Manuscript. Word and code examples are just a bad match anyway. Manning are fixing this and switching to a DocBook based XML, including a way of including sections of external files as code examples. There will be some markup for styling, but even hand-crafting XML will be less painful than fighting Word styles.
Omit unnecessary words. Omit unnecessary words. Words used for emphasis are usually superfluous; very, actually, basically - I'm talking to you.
Whenever you use 'it' or 'this' to refer to a concept it is possible that the subject is ambiguous. Many occurrences of 'it' should be replaced with the subject they refer to; repetition is better than ambiguity.
Omit unnecessary punctuation. Quotes and exclamation marks should be used sparingly. Parentheses should be used sparingly, a rule I frequently break. Dashes to break sentences should be rarely used.
I am terrible in my use of commas and semicolons. I cause pain to non-native (and some native) readers of English. I have improved a bit, but still don't see the problem in many cases. sigh This is one area where copy-editing was particularly important.
The big up-front design employed by Manning for sketching out the book section by section, their proposal process, really didn't work for me. The overall structure chapter headings and some of the section headings were useful, but the detail changed so much in the process of writing that much of the initial effort creating a detailed proposal was a waste of time. Next time I would detail maybe one chapter and only outline the structure of the rest of the book. YMMV. Having a completed book under my belt will make it easier to elicit a second contract even if I refuse to jump through all the preliminary hoops next time...
Some people like extensive footnotes, some don't. You can't please everyone.
Some people prefer real-world application sized examples and some people prefer recipe style examples. With a decent sized book you can please everyone.
I found that the hardest parts are the introductions and the conclusions. This makes the start of the book horrible to write. Manning have a foreword not written by the author, followed by 'about this book', 'introduction', 'part 1 introduction', 'chapter 1 introduction'. That's four introductions in a row.
Creating the index sucks big time. Having someone who knows the subject (i.e. the authors) create the index is important if it is to be relevant; but it is such a mind-numbingly painful process that I'm not convinced it is much better than an automated attempt.
The publisher won't do much marketing for you. Podcasts, interviews and reviews will be your marketing lifeblood once the book is published. Having a book website helps. A website also has the advantage that you can link to the book at Amazon, or directly with the publisher using affiliate links. If people buy the book through your affiliate links it can almost double the amount you make on that sale...
The Manning Early Access Program (other publishers have an equivalent) where the unedited early chapters are available to readers who buy the book, was invaluable. Early feedback and typo corrections are the two big benefits.
Your early reviewers won't be able to help themselves but complain about grammar and punctuation even if asked not to. Grammar and punctuation are mainly fixed in copy-editing, which is one of the last stages - so it isn't the feedback you want from your early stage reviewers. Your reviewers will contradict each other wildly of course. Pick out the comments they agree on or the ones that particularly strike a chord.
The technical review, which with Manning is a separate stage to the other reviews, is also invaluable. We were extremely lucky to get Dino Veihland (core IronPython developer) as the technical reviewer. I didn't agree with all the comments and changes he suggested, but it was all helpful.
After the book is finally published the only indication you have of how well it is selling is watching the Amazon rankings like a hawk. It gets boring after a few weeks, but you can pay a small amount of money to Rank Tracer and have them monitor it for you - or you could hack up some scripts. I have read two different blog entries from authors (links long lost to the vagaries of the web I'm afraid), one saying that a good technical book should hover around the 20 000 rank and the other saying that a good technical book should hover around the 100 000 rank. After three months IronPython in Action is still hovering somewhere between the two.
It is hard not to feel competitive about other books on the same subject, even if rationally you know you shouldn't...
I've blogged about the sales figures and royalties from the first and second quarters:
The norm is for you to receive an advance, which will vary hugely depending on all sorts of factors. Manning say as a rule of thumb they guesstimate sales and aim to pay half the total expected income for the author as advance. Manning advances are paid in three stages as the book is being written - one third each when the book goes through the first and second reviews and the final payment on publication.
Royalties are paid based on the gross amount the publisher makes from selling the book . If Amazon are selling a book for $30 then the likelihood is that they paid the publisher about $15 for it - and the royalty percentage comes from the amount the publisher gets. Once the book has sold enough for the royalties to pay off the advance then the author actually starts to earn [some] money. Many tech books never sell enough to pay off the advance, making it the only money the author will ever see for his pains.
I should have been braver when negotiating the contract. It is a tough thing to do on your first contract and I bottled it.
- I didn't realise that publishing print ready images and diagrams was my responsibility. I discovered this after the contract was signed - and was told that if I wanted them to turn rough diagrams into print ready images then the cost would be deducted from future royalties. I'm not sure this is specified in the contract (which I did read in full), but as the awesome Jonathan Hartley stepped into the breach and turned my awful (Microsoft Paint) scratchings into fantastic diagrams it wasn't an issue. It could easily have been a very big issue though.
- I didn't really realise the implications of being offered the same gross percentage on dead-tree books and e-books - but obviously the production and distribution costs (the marginal costs) for an ebook are zero and there is no reason for the publisher not to offer a much higher percentage. I believe that some publishers offer 50% on electronic sales, which sounds reasonable to me.
- 10% of gross is not bad - and is relatively standard or even good for a technical book for a first time author. The money is so not worth the effort that I won't be writing another book unless the rate is higher. Apress (for example) offer a tiered rate  based on how many you sell, which sounds like a good incentive to write a good book. If Manning wants my future books (and despite the points I raise here my overall feeling about Manning is still good) the bottom line needs to be further away from the bottom.
- The copyright on the book is still owned by Christian and me and if the book goes out of print we have the right to do what we want with it; i.e. put it on the web. This is great in theory, but in practise Manning could have five copies in a warehouse and still have the book be 'officially' still in print. Specifying a time limit after the last edition would have been better (five years sounds reasonable). In fact for future works I would like to retain the rights to publish the content free on the web as well and I don't think it would hurt sales. The best way to force this to happen is to have the material already on the web free and if they don't want to publish it then it's their loss.
- Manning don't yet publish books for the Kindle. Not much of a deal (yet) and not really a contractual issue but worth mentioning. Their PDF ebooks are of excellent quality and allow copy-paste etc which is better than the DRM crippled ebooks some technical publishers produce.
- The book is available via Safari online. This is a good thing of course but I'm not sure if I'll see a penny from this. I need to look into it. (Manning assures me that Safari sales count as domestic sales.)
- I stated up front that I would have nothing to do with making the book 'print-ready'; that layout and everything associated would not be my job. I should have got this written into the contract. (See the earlier point about the diagrams for example.) I spent a non-trivial amount of time learning to work with the Manning templates and styles so that book could become print ready and although Manning needs this doing I consider it wasted time on my part.
- Try to get out of the responsibility of creating the index if you can. It will suck less if you do it but it is a horrible task.
- Watch out for how royalties for translations and future editions are specified. Manning's contract is ok but some publishers just specify 'some money' which could mean anything.
You have much more power when negotiating the contract than you think; even as a first time author. Publishers generally have many more ideas for books than they do willing suckers to write them. You can write a book and self-publish it without a publisher. Publishers can't market and produce books without authors. If a publisher is interested in an idea you have for a book but won't agree to these terms then tell them to forget it and save yourself an awful lot of pain.
|||Despite having spent two years on this process you'd be amazed how quickly the memory of the precise details starts to fade. Something about the brain blocking out traumatic experiences...|
|||The only exception to this that I have heard of is pragmatic who offer 50% of net. Normally steer miles clear of net deals as the publisher can always produce a million costs associated with the book as a reason to never pay a penny. Pragmatic are probably honourable though, at least until they get much bigger... (Anyone remember don’t-be-evil as a company slogan?)|
|||My understanding is that a typical Apress contract starts at 10%, going up to 14% dependent on sales.|
For buying techie books, science fiction, computer hardware or the latest gadgets: visit The Voidspace Amazon Store.
Last edited Tue Aug 2 00:51:34 2011.