Publishing with Silicon

I was on the phone with a prospective client today, who shall remain nameless and unidentifiable. This could be any company, as they face the essential predicament of anyone trying to get the same content to go to both web and print effectively.

On the one hand, there is so much commonality and re-use of their content across the web and print media, it is absurd to have two entirely different workflows. On the other hand, the tools that lend themselves to a real multi-channel workflow, such as real XML content management, take extreme effort and time to implement and often have expensive associated software. Even after that effort, authors or content sources may not fit in with the required content process at all. Beyond that, moving content over from an unstructured to a structured format can be really difficult.

Inevitably, XML demonstrations make business users underestimate the challenge. “If you show us something, show us with our content!,” he said; evidently they were shown a rosy picture where perfectly marked up XML flowed easily out into web, print, braille, video, whatever. It is true; if you have rich semantic markup the publishing capabilities are amazing.

The challenge is getting that richly marked up content. It is hardly automatic. The extreme best case for authoring such content is the world of technical documentation, where the authors are typically really technical, and highly-evolved schemas/toolchains like DITA give them guidance on how to structure content. But at the other extreme, with writers who are non-technical, it is hard to get them to work with tools that are too constraining, or to get them to follow rigorous guidelines. No pain, no gain:  without the rich markup, publishing becomes more of a channel-by-channel basis.

I believe over time things will get easier, with standards like DITA, greater support for XML authoring in tools, and better example workflows for organizations smaller than the Department of Defense. But the pace of such improvement is slow.

We have built solutions using InDesign Server since it came out, and before that we were building solutions based on InDesign desktop for 5 years. So we know the XMPie space pretty well.

XMPie is a really well-built program, that to me has three main benefits:

1. It lets you easily define a data source for variable content (using uPlan) and reference that data source directly in InDesign (via uCreate)
2. It manages XMPie jobs (via the uProduce server), with functionality exposed as Web Services
3. It optimizes print output, producing VPS (which has been known to work), PPML, “VIPP” (which is known not to work; it is not VIPP but a VIPP wrapper around PostScript), etc.

XMPie is salvation for the designer at a mail house: they can bypass Programming entirely and set up their own “campaign” based on new InDesign/data input from a client.

Yet these days, we never meet such a designer. We meet enterprise clients, who consider themselves very special and do things a very special way. XMPie invariably meets their needs 40-80% of the way, but the other 20-60% can take a supreme effort. So we need to request extensibility from XMPie, and in many case fuse together an XMPie workflow with a very non-XMPie workflow. They may be sick of my requests, but they have given us more and more extensibility over time.

InDesign Server has to its advantage complete flexibility, but if you use InDesign Server alone you have to build several features that are pre-existing with XMPie. It really depends on specific workflows/document types/staff whether XMPie is the right fit.

We worked with Adobe a bit on their Scene7 product, and I have to say that it is some of the most promising technology out there for Web to Print. There are two big gotchas that I hope are overcome soon:

1. The text that is possible with Flash 10 is not fully functional: this stands to improve once FXG 2.0 is available, the hope is that FXG 2.0 will be fully supported. As of now the text is more like FXG 1+, it isn’t quite robust enough for our typical clients.
2. The pricing model is crazy. I think they priced it so high that they would make sure not to get slammed with too many initial implementations. $50K/year as a base price with multiple forms of transaction/bandwidth costs on top of that is hardly a SaaS model. You either pay as you go or you pay up front, they can’t ask for both…

Also, it appears in their early concepts of how the app would be used, they imagined one would hit the server for the Flash renditions! I think the whole beauty of sharing the XML model between PDF and Flash is leveraging what the client can do…

Demonstration of the Scene7 web-to-print solution

Anyway, in all my years of working with great programmers, including many at Adobe, I have never seen a group as great as those working on Scene7 web to print. I am very optimistic about its future.

The fundamental beauty of the Scene7 model for web to print is that it uses the same XML to describe the web document and the print document. It also extends the XML used in Flash (FXG) to support requirements of print such as CMYK color. Tricky, as this would ideally not be done in a separate namespace, but would be part of the core FXG spec itself. In general it is awkward how the different groups at Adobe work together: they are all focused on their own short-term deliverables and can’t often reconcile or coordinate the overlapping parts of their efforts.

Which brings us to… InDesign Server. One might have guessed that Scene7 would use InDesign Server rather than build their own form of PDF generation totally independent, with a different text engine (common with Illustrator/Flash, not InDesign) and different XML model (FXG vs. IDML). Sadly, the InDesign project does represent the ultimate in text engines, the ultimate in document feature sets for long documents, etc., but there has not been a desire to use it from the Scene7 group. They didn’t find it much of a true server product, apparently, which is quite understandable. The “server” dimension of IDS is minimalist, it is essentially the rendition half of the desktop product with a few hooks and enough “build it yourself” aspects that solution providers like us have a fairly endless stream of opportunity.

So Scene7 will hopefully become a big part of our work next year, assuming the 2 issues above are handled, yet InDesign Server will remain, especially for longer documents and those cases where extending a desktop InDesign workflow to the server is easier when avoiding issues around reconciling text and layout engines. We don’t really mind two systems, but some day I’m sure we’ll hit a hybrid case where we use both, and in some long-term road map (CS7?) they should actually get reconciled.

Adobe product managers have managed to calm down my early complaints about non-reconciliation of these two engines. One pointed out the incredible backwards compatibility responsibilities of IDS: they can’t just start over… one tiny bug in one tiny dot release can screw up a million documents for a client, they are not as agile as a SaaS shop.

In terms of SaaS; as of now, Scene7 is almost only SaaS and IDS is almost only self-hosted. It is likely that both products will cross over the other direction. We can host IDS in an EC2 environment just fine, with great scalability, yet the licensing is not SaaS friendly. In similar fashion, Scene7 can install just fine as a self-hosted software, yet they only allow this in “special” situations and tend to push for SaaS at all costs.

I just set up a basic WordPress blog for Paris Tompkins. This took only a few minutes’ spare time, including the hosting, DNS, customization. She chose the theme herself; I just made the side look OK and added the vital “Add to Any” plugin.

I still barely know WordPress, but I have had almost no trouble with any aspect of it recently: I played around with various blogging software about 8 years ago, and it was nowhere near this easy. I have only had to do something with PHP within a WordPress site once, and probably because of the nature of the template.

I am an impatient person and I will never be content with the pace of publishing technology, but I have to say that when you look back to where the web was in 1996 and where it is today, it has generally gone the right direction. Well, that is after starting out with a fairly complete misinterpretation.

Proposal for the World Wide Web

The original concept was of course brilliant, and you should check out the proposal for the web if you haven’t already. Basically, the intent of the web was to facilitate two-way communication, but of course the mindset of a Television-soaked population at first thought of it more like a single-direction, one-to-many, broadcast medium. We only understand things in relation to what we’ve seen before, at least at first.

So the Web started out with much knee-jerk reproduction of the Television model, and only with the gradual evolution of blogging and social networking (and concurrent evolution of tools for this) has it become really easy for a person like Paris to get out there and express herself. Now even the large corporations are hyping Facebook and Twitter, with their own YouTube channels and real estate in SecondLife. I think Paris will fare better than most corporations, because the playing field is leveled.

Adobe has finally defined an XML format for Flash, something those of us in the SVG World have long waited for. Well, we weren’t waiting for Flash, it was an XML server-based description of interactive graphics, but who’s counting?

Now that we have XFL and FLA renaming it, can we get a server? I really look forward to this but it remains completely unannounced.

I was wondering why a dirt-simple InDesign file packaged to 6 Meg… and I discovered that the images were about 1 Meg each. I had created very simple images, colored rectangles in Illustrator CS4 and saved as straight .ai files with default settings.

Amazing, what valuable info is it persisting that makes this the right size for a primitive image? When I export as SVG I get 6 lines of code, which look verbose already, but way tiny by compare…

Does anyone remember the quote attributed to Bill Gates that PCs would never need more than 640k of memory? Poor Bill could only see the lower 2/3rds of this rectangle with the dream computer of his age. The only thing more amazing than the continuous exponential increases in memory, disk space, and bandwidth is the way such improvements are consumed by new applications the moment they are available.