Lois & Clark Forums Forums Fan Fiction Fanfic Related Fanfic in epub formats - archive anywhere?

*waves hi and grins sheepishly* I'm actually poking my head in here, after I don't know how many months. I had a question I wanted to pose.

See, I got an e-book reader recently (Astak EZ Reader Pocket Pro, if anyone's curious which), and its best format for reading is epub. Although it can read txt files just fine, testing the Archive's files brought me to a screeching halt.

A small minority of fanfics on the Archive, when saved, have so many words and then a forced line break. This is easy to read on a computer screen, since my computer screen is far wider than the line is. On my e-book reader, on the other hand, the screen there is smaller than the line, forcing some text to drop below. This requires more page turns (due to the inefficiency at displaying text) and is more challenging to read.

The vast majority of fanfics there are completely unreadable when saved, even on the computer, as the line breaks are replaced with "undisplayable" characters (i.e. those little boxes like when you have Chinese characters and don't have the font sets installed). Text wraps continously with no line breaks whatsoever, and every paragraph is only separated by two boxes, with more boxes in the middle of sentences where the original forced line breaks were.

Now, I understand why this is the way it is; when read online at the Archive, the fics are very readable and exactly as designed. However, since I'd like to read some fics offline, I'm going to have to go through them and edit all those line breaks out so there are just paragraph separations. While I'm at it, I might as well make them epub format, which is basically xhtml in a zip container; it's very small file size. I took a children's book and converted it just the other day, and the epub size was less than half of the txt size (122 kb to the original 267 kb). Epub has another benefit in that it lets you create a TOC and jump to individual chapters.

Since I'm going to do this for various fanfics, is there any way the Archive would want to host epub versions of fanfics as well? Should I post them somewhere online? Since this is simply re-formatting (with some chapter breaks--which I can use as they were posted on the mbs or just find scene breaks every so many pages, roughly), I don't see as big an issue with it as with audio fanfics, or translations, but would there be permission issues? Do I need to just keep it to myself, or is it something I could share with other FoLCs who also have e-book readers and would like nicely formatted epubs they can read offline?

I would welcome any thoughts/comments any of you have on this, particularly staff input as much of this is a question only they can truly answer. (I'm also curious how many of you read fanfics on an e-book reader and would find this sort of thing useful. Anyone else already doing this type of thing?)

Hi,

Have you tried using Stanza to change the stories into ePub format? I have used Stanza's desktop program for the conversion and then read the stories using the Stanza app on my iPod Touch. I have found the stories so converted to be extremely readable. I have never had to tweak line endings at all. (The current version of Stanza doesn't even require translation into the ePub format; but I had used that format with an earlier version of Stanza, back when its app it enable file transfers over a wired LAN via a separate third party program.)

Joy,
Lynn

I use Good Reader on my iPod Touch, works really well. Can read stories in many formats, including PDF, plain TXT, Word documents, etc.

I briefly considered trying Stanza, but I don't have an iPod Touch or anything of the like, and I couldn't figure out where to download it. I'm also not fond of anything too Mac-centered (I actually figured out how to install just the necessary files for my iPod Nano to communicate with my computer without installing iTunes itself, which is a memory hog on Windows and not a program I want at ALL--I'm using foobar2000 to manage my iPod library instead).

I poked around, and ended up going with Sigil, which had such a shallow learning curve that I've got one children's book converted and am working on a second, in about two days' time, TOC and all. I haven't tested it on the fanfics yet, but I plan on editing those line breaks in the txt file anyhow so I don't have them hanging around all uglified. If I try it in Notepad++ or some other Notepad replacement, I hope to be able to do some better search&replace, since Notepad won't let me replace boxes with carriage returns.

The Astak EZ Reader can read a whole lot of formats too, actually--that was a big reason why I wanted to get it over other e-book readers (price was the other one)--but not all of them have all the features. Some you can't search in, and a lot of them don't have a TOC or anything like that. PDFs are close in features to epubs (though some look atrocious due to the nature of pdfs), but epub is better than the rest because it has search, TOC, and more zoom levels altogether. Txt format is very limited on the Astak EZ Reader, so I would need to convert to something else if I wanted to take advantage of any of the possibilities (TOC would be nice with some of the gigantic novels--I can't imagine trying to read Masques, for instance, without some sort of chapter system--what if I wanted to just read from a certain point on?). Epub is a natural choice because it's the best-supported format on the Astak, and Sigil is an easy program to work with. I can make sure it will look good and be formatted nicely when I'm done.

The desktop version of Stanza quickly and easily converts documents; you don't need an iPod to use it. It's available at http://www.lexcycle.com/desktop . To convert a document, all you need do is open the plaintext version of the document go to File, choose "Save As," and select the name and location you wish the ePub version to be placed. Couldn't be easier!

But it sounds like you have some other ideas in mind. Whatever you go with, I hope you do find something that will work for you.

Joy,
Lynn

This may help you, Doranwen. I've got a Sony Reader and it uses "PDF file, EPUB file, Text file, RTF file, Word file (unsecured)*1,
BBeB file".
Also I'm on a PC with System 7 right now.
For the Archive files I convert them to Word to read on my computer and then convert them to RTF to read on the Reader. There are no line break problems or format issues there. It works just fine. I'm reading "Aussie Rules" on it now. True, that's from these MBS, not the Archive. But the same thing works.
It's simple, it's easy and doesn't require anyone's permission. Give it a go. File size doesn't seem to be an issue with my reader.
regards
Artemis

I think there's a bit of confusion here about what I'm asking.

The Astak EZ Reader supports over a dozen formats, including TXT, RTF, PDF, DOC (though VERY poorly! I won't use it because it treats font sizes as pixel sizes, necessitating a major font size change document wide), and, of course, EPUB. I don't need to change the file format to be able to read it in the first place.

However, I would like to have TOC with the larger stories (I gave Masques as an example of a size that would be very handy to have some sort of chapter system to jump to if I wanted). Txt might support search (haven't tested it), but not TOC. It also has an ugly-looking font compared to the epub standard font, so I'd rather not keep files as txt. Rtf is not the best format to use either, since it lacks TOC as well, I believe. As epub is best supported by the Astak, I am choosing to convert to epub. I will probably do this for my own personal enjoyment whether anyone else does so, or whether I can share it or not. File size is not a concern on my end as I have a very large SD card that the reader accepts, meaning I can store my entire library and then some, when it comes to stories to read.

I do have some issues with converting line breaks, and those will be issues whether I copy to Word or just edit in the txt file. Even copying a fic straight from the Archive to Word does NOT eliminate the forced line breaks, meaning I have a lot of work to do to make it flow across each line naturally until the paragraph ends. Most saved txt files replace those line breaks with strange boxes, and I can't imagine any program fixing those automatically. If Stanza does, that IS interesting, but it doesn't appear it is a program I can use. Doing a search a little while ago seemed to imply that one had to get it through iTunes, and I don't plan on installing iTunes on my computer for any reason. The best solution would be to work on some sort of macro that would do the editing for me. I'm not so good at setting those up, but with the amount of fics I'd like to have at my fingertips in epub format, I think it would be worth it for me to learn. That might be the subject of a separate post in Off Topic.

Anyway, since I plan on creating epubs for my personal reading anyhow, the real purpose of this post was to ask if the epubs I create this way could be shared somehow, whether anyone would WANT a copy of them, whether it would affect permissions, etc. I mentioned the very small file size because if someone were to host them, small files are always preferred over larger ones. Knowing they take up less room than a txt file (which is, admittedly, smaller than most since it only encodes a byte per character, with no overhead unlike Word's 31 kb overhead) is helpful in that respect.

It is interesting to hear that the fics seem to be very readable to the rest of you as-is. I haven't figured out how that works, unless the forced line breaks are just right so they barely are noticeable. In my case, the screen I read on is somewhere about 2/3 (I think?) of the line, requiring a lot of alternating full lines and short lines of just a couple words. This is distracting and difficult to read, plus it stretches the file out to many more pages, requiring much more frequent page-turning (and I'm already a very quick reader!). This is not ideal, and worth the effort to fix the files, in my opinion.

Thanks for the input so far!

To get rid of the undesired line breaks:

Cut and paste into word.

Search and replace - ^p^p with @@@ (or some other random thing that won't naturally occur in the doc).

Search and replace - ^p with either nothing or a single space (do one with no space before the replace all and see if where it replaced it you have twowords with no space - if so, you need a space; if not, no space necessary).

Search and replace - @@@ with ^p^p

Should fix all the bizarre ones without undoing the extra hard return between paragraphs.

No idea how to help you on TOC, though. Have you tried Calibre? It...*looks* like there's a TOC option, but I don't know how to work it.

Good luck!

Bethy

Hi Dorenwen:
Yes, I did understand what you were asking.

regards
Artemis

Artemis,

Ereaders and Word will do the word-wrap correctly in an .rtf file if the line is too long. If I understand correctly, Doranwen's problem is where there are extra hard returns, causing some lines to be too short.

For example the paragraph would look
like this
and then keep going for a full line
and then
another short one, etc and so forth.

(With the original having been:
For example the paragraph would look like this
and then keep going for a full line and then
another short one, etc and so forth.)

It's the extra hard return after *this* and *then* that are causing the problems.

At least, I think that's what she means - Doranwen? Am I right?

Bethy

Yes, Bethy, I did understand that. I did various experiments and found it didn't bother me enough to go to any effort to deal with it.
Whether the Archive can be put into epub is up to LabRat and crew.

Artemis

Aha! I didn't know that ^p would represent the carriage return (though now that I think about it, I should have known there would be *something*). YES! Now I can fix them. Thank you SO much, Bethy! (The fics do need that space replacement, I found, so I'll remember that for the future.)

I do have Calibre--I use it for quick conversions when I just need something readable and searchable and I don't care about it having a great TOC. It will create a TOC automatically if the file is formatted just right--I'm not quite sure how to do that, though, and it's a bit touchy about it.

As for the TOC, Sigil creates them very easily. The only thing I have to do is mark headings on the text. It'll even do multi-layer depth to a TOC. For instance, I could have a Part I and Part II, and under each of those have separate chapters, so would be able to choose which part, then which chapter. I also like the polished and finished aspect of epubs better.

And yes, Bethy, that's EXACTLY the problem--you illustrated it perfectly. My screen is small enough that I had major issues with it. I suspect that my fast reading pace exacerbated it beyond what would frustrate most people. I timed myself on Long Strange Trip once, actually, and it took me only about 4 1/2 hours to read the whole thing. At that sort of pace, the constant flicking of the page turn is not only annoying, it gets tiring on the fingers AND drains the battery quickly. Fixing the carriage return problem will do a lot to help. (And converting to epub will give me a nice TOC and a better viewing font. )

Having all the technical ability of a drunk wombat, you lost me after e-book, Doranwen. .

But I will pass this along to Lauren - who has all the technical ability of Rodney McKay - and see what she thinks.

LabRat (quite pleased with herself to have managed to work her current Atlantis obssession into this post....)

Glad it was helpful, Doranwen!

The carriage return is something that bothered me even on regular computer screen reading - in that every line was too short. I can't take full credit for the fix though - someone else on the boards, at some point a long time ago, explained how that works.

So I was already doing it and then when I got a nook, I kept doing it, because yes, the problem is exacerbated on the smaller screen.

Also, just as a note in case you get fic from other websites - some don't have a hard return but a... "soft return"? I think that's what it's called. In Word, you create it by holding down shift as you hit enter. If you turn on the paragraph/space/etc indicators in Word (a little paragraph sign P on the menu), it'll show up as an arrow that turns at a right angle (like the one on your enter key).

IF you get that, you need to replace ^l (lowercase L) with ^p. Yes, it took me hours and multiple friends to figure that out. I nearly cried before we got to the solution, because I had just pulled some long fic that I really, really wanted to read on my nook and it was full of the ^l returns.

(ETA - the ones I was having a problem with were not mid-paragraph, but the normal carriage returns at the end of paragraphs. The problem was, the nook ignored them altogether and so I ended up having one solid massive humongous paragraph. Just...in case that shows up differently/makes a difference to you.)

And Labby - lol at the working in your current obsession. Nice job!

Bethy

When I saw this thread, my thought was. "Exactly! I would love to have the stories available in epub format."

But then as I read the thread the replies all seem to be about how to convert txt files to epub. I have done that with... I don't know, somewhere about 50 to 100 stories. I convert them for my ebook reader.

So, let me try a more direct question:

Would it be possible to start an effort to convert existing stories to epub and set up an epub section on the archive?

This would be ideal for me. I'd be happy to work on the project and donate the ones I've converted. At the very least I have all of my own stories in epub.

So, I'd be curious to hear some reaction to this.

Bob

Re the issue of getting a TOC for a story, for Word 2007 and later there is a "Document Map" on the left side of the window that is a TOC to any level you want.
Bob, since you've converted so many txt files to epub, I think folcs on this thread would be interested in how exactly you did that.
regards
Artemis

Re: I don't have any magic. I use the Stanza desktop application on my Windows PC.

If I want a quick-and-dirty conversion, I load the txt file and do a Save As to save in epub foramt exactly as Lynn describes above.


If I want a clean conversion, I do this:

1. Load the txt file into MS Word

2. Use a technique almost exactly as Bethy describes to remove the hard-returns. Any yes, you need to be ready to handle the soft-return variant. It's just a different search but once you find the character it works perfectly every time. (And my experience in figuring this out was very similar to Bethy's.)

3. Then I have one more trick. For each section, add the word "Chapter x" to open each section. Then save the whole thing as MS Word format. (The is partly why I started using the word "Chapter" in my longer stories. It saves ME a step )

4. Then load the .doc into the Stanza desktop app. About 2/3 of the time Stanza will see the "Chapter" key words and create the chapter breakpoints. I don't know why it doesn't work all the time. I haven't tracked that down yet. If someone knows the magic for this part, I'd love to know.

Then Save As the epub. If Stanza recognized the chapter breaks, you will have a properly formatted, chapterized epub version.

Yes, this takes a little time but once you've done it a few times, it's pretty quick. I'd say less than 10 minutes for most files. The hard part is breaking into chapters if there isn't a good substitution word.

Bob

I've used Stanza on my mac but haven't tried to add a toc to anything. However since you say it works most of the time but not all it is probably something like needing 2 newines in a row and starting in column 1, and maybe having a newline right after the number. Two newlines in a row is a common this starts a new paragraph marker.

You might see what is a good Windows text editor (not word processor) and look at the file in a mode where it shows every character even the "invisible" ones like newline, tab etc. Than you can see what the difference is between ones that work and ones that don't.

Yep. I tried all that and more. I figured it had to be something, but you reach a point where you just have to shrug and admit that it isn't worth it. I may take another look at the file with a binary hex editor and see if I can crack the code, but I haven't had time yet.

Edit: Sorry if the above sounds flip. I just got frustrated that I couldn't find the pattern. I'm pretty sure that I looked at all these variants and others. I usually figure these things out in short order. I don't like the fact that I don't have the answer.

Bob

Oh, Sigil is so easy to use that I wouldn't want to do it in an auto-conversion format (Calibre has turned out very questionable contents--very little TOC recognition overall, too). Sounds like Stanza doesn't always recognize the Chapter markers, whereas I can name a chapter anything I feel like (granted, in fanfic, it'll always be Chapter 1, Chapter 2, etc. unless the author had fancy names for each part), and just apply a heading format (H1, H2, H3, etc.--H1 is usually for titles, H2 for author, H3 for chapters, and H4-H6 for subheadings if needed). Sigil automatically recognizes those, and I can go into a TOC editor (a simple dialog box showing the list of chapters) and uncheck any I don't want as sections to jump to.

In the last three days since I started using it, I've created epubs out of two children's books (they required extensive editing to fix scanning errors, basically entailing re-reading them), and today, when I didn't need to do major editing beyond replacements, I created epubs out of one book, two poems, and a short story, in between all the cooking I've been doing. That one book only required a simple scan for each chapter marker, insert chapter break, highlight title and mark heading, and go to the next chapter to get a TOC formatted nicely. Very little work and produces an excellent result. So I could create tons of epubs that way, if the Archive would want to host them. I'll probably create a good lot *anyway* since I want to be able to re-read all my favorites off-computer (and I have a LOT of favorites!).

And aside from fixing the line break issue (which I can't thank Bethy enough for), my real question was indeed whether an epub option on the Archive (kind of like the audiofic option, perhaps?) is something that might be done. It would appear that there are a few of us willing to create these epubs and format them nicely for the Archive, which only leaves the question of whether the Archive would want to store them. Obviously there would need to be some new coding to fit them in, so that creates more work for people . . . I think it would be worth it, though, since more and more of us have e-book readers which read epubs, and computer screens are so bad for the eyes if you want to sit there for hours glued to those addicting epics. (Forget printing them out for real; I would still love to be able to hold some of them as books in my hands, but until I get equipment to do my own print-on-demand stuff, which will probably happen the day pigs learn to fly since that takes a lot of money I don't have, that isn't a realistic dream.)

Hi, Doranwen,

I really like your idea of making epub versions of stories available. I've been reading fanfic as epubs too, using Stanza on my iPod Touch. I second everything Lynn S.M. says about Stanza. Fantastic app.

Your description of Sigil is intriguing. Just tried it. Wow! With the fields for editing metadata (title, author), it's nicer than desktop Stanza at creating epubs, in my limited experience -- but Stanza is aces as a reader.

Regarding your problem with line breaks, there's the Word solution others have described. You could also give the Formatter page on the archive a try. It converts among fluidly wrapped text (as in your word processor), archive-formatted text and simple HTML. You can strip out the extra breaks using it. It'll work on stories using our current format, which has a blank line between paragraphs. For older stories that use tabs but no blank lines between paragraphs ... not so great.

Though when I opened an archive text file in Sigil, it knew what to do -- just like Stanza, it stripped the extra line breaks, and all paragraphs were fluidly word-wrapping. AND it did it for both old and new stories with their varied formatting. So, no tedious manual line editing or search/replace routines required. It thought the header block was one paragraph (like my formatter script does), but you can easily "press Enter" to restore the line breaks you do want.

Both Stanza and Sigil fail at poetry formatting -- if the lines of a poem's stanzas are single-spaced, they get word-wrapped into one paragraph. So poetry and song lyrics would have to be manually edited, just like the story header block.

There is one really nice feature desktop Stanza offers over Sigil: It lets you export a story in various flavors -- HTML, Word, plain text and PDF -- in addition to epub. It can even export to the mobipocket format, which Amazon's Kindle supports. (The Kindle, probably the most popular ebook reader of all, doesn't support epub, strangely enough.) Looks like Sigil saves as just epub. You might try using desktop Stanza for your intermediate conversions.

While I think it would be great to host epubs on the archive, the logistics will take some working out. Short-term, maybe a file uploader tool to let you upload your epub files to the server? Then maybe a wiki page you could edit in order to create download links? That would make them available to readers fastest. It sounds like bobbart and maybe others would like to contribute too. (Thanks, bobbart!) This could be a workable solution for all, though it may take some coordination to prevent duplicate stories being posted.

Long-term, we could put the epub download links next to the main story links in all the catalog pages. That'd take some time to pull off.

For a sample of what this could look like, see the archive's Filename Z page. As one of the shortest catalog pages, it makes a great sandbox for testing. Aren't the icons lovely? Just added the epub versions courtesy of Sigil -- they're the black buttons at the right of story titles. (Firefox is trying to open them if you left-click on them, but right-clicking and "saving as" works to download them to your hard drive.)

Filename Z is a prototype of what I see as the future of the archive: The site would be database-driven, with HTML the default story format, and text, OpenOffice, PDF and epub flavors (thanks to your suggestion) also offered. Maybe mobipocket too. But imagine nicer icons for the different story formats.

My nebulous hope is that if we feed a word processor story file into the future database, a magic conversion process will kick in, creating all these different formats. Getting to the magic part is what has me stumped. I've been plodding through PHP books, trying to wrap my English-major brain around programming logic. But it seems doable. OpenOffice format, for example, is basically a zipped XML file. Couldn't styles designated in an OpenOffice template match up with CSS styles for HTML pages, and even for epub docs too? Headline tag to headline tag, author tag to author tag, etc. And tools exist in PHP to manipulate XML documents and convert them to PDF and other formats. There's got to be a way, without doing it all by hand. If a story is properly tagged going into conversion, it ought to work. (She said blithely.)

An automatically converted epub, if that's even possible, would be missing the value-added features only a human could provide, though -- the chapter breaks, TOC and appropriate poetry/lyrics formatting. A bonus of going database, besides more powerful searching, is eliminating hand-coding. I suppose some will always be necessary. It's all hand-coded now.

Sorry, got off-topic.

Permissions in that authors' stories will be converted into a format they didn't anticipate, or permissions in that you'd like to host the files elsewhere?

I'm perfectly happy for us to host epubs on the archive, and I'm equally as happy if you'd prefer to post them elsewhere and have us offer a link to your page. But I'm fuzzy on if there's an implied agreement between the archive and authors about hosting their stories. When they submit a story to the archive, they know it's going on the archive. Would they be upset if it went other places too? Authors who happen to be reading this, would you object? And how about story formats: Would you mind if your story were turned into HTML, PDF, OpenOffice and epub?

A precedent may have been set for this years ago. Soon after Rhen's collection of FTP story files turned into a Web-based archive, someone out there in the ether created a mirror site. An Australian guy, I think. Nobody objected as I recall. Haven't heard anything of it in the last 10 years, so it's probably gone.

But I do remember we lost some authors and their stories in the process of going from a sheltered FTP site to a very public Web site. Because they were using their real names, they felt exposed. And for one reason or another, they were unwilling to adopt a pseudonym. One author -- Gail L. -- wrote a great story about Lois and Clark being stuck in an elevator for a few hours. I lost it due to a hard drive crash, darn it, along with all her other stories. Not that this last relates much to the issue at hand, other than that people sometimes offer objections you don't anticipate. (And I really miss stories when authors pull them off the archive.)

In the case of audiofics, as you say, it's a bigger issue. There was a thread where authors opted in to give their consent to have their stories recorded. But epubs are not that radical of a transformation -- it's still text you read on a screen, though more attractively formatted.

I'd say permissions issues are LabRat's call. (Tossing it back to you, LabRat! :p )

Thank you, Doranwen, for poking your head back in. I think it's an exciting idea, and clearly there's lots of enthusiasm.

And thank you too, bobbart, for your interest and the Stanza how-to.

So what's the next step? I can start looking into a file-upload script and prepping a wiki page if that's agreeable. Or we can talk more and explore other options. I think it'd be good to hear LabRat's thoughts and see what our authors have to say.

Best wishes,

Lauren

P.S. I had to go to Wikipedia to find out who Rodney McKay is. Thank you for the comparison, LabRat, but I fall way short! He's probably more like bobbart, who can use a binary hex editor. How ubercool is that?

Great post, Lauren. I didn't know about the different formats on the Archive. There's been a poll for a long time on the Archive asking if people want stories in HTML. Maybe there's an indication there of interest.
I have no objection to my stories, few that they be currently, being posted in other formats. I suspect that if someone from this board posted them elsewhere, I would have no objection either.
But I went to FilenameZ like you suggested and saw it in PDF. I'm in hog heaven now. PDF goes directly to my reader and works perfectly, so I'm happy with that. PDF has the advantage that no receiver of the file can make changes to it. Though why someone would want to change the stories, I have no idea. We get a lot of secure mail (travel reservations) in PDF.

Thanks for all your hard work over the years, Lauren. And all this from an English major! I'm impressed.
much regards
Artemis

Hi Lauren! Your vision for the Archive vNext looks great.

The whole talk about automatic document conversion certainly causes some PTSS flashbacks over here Fortunately for us, novel-type text is quite simple, as far as formatting goes. So, yeah, as soon as you have the document broken up into paragraphs, the rest should be quite doable. At least in C#/.NET.

Michael

It also has the disadvantage that it's designed to display things exactly like the original. This is not a good thing when it comes to e-book readers, since their screens are considerably different sizes than a computer screen (the Astak does a fairly good job at text reflow on pdfs, but it's just making the best of a badly suited format). PDFs are considered one of the very worst formats when it comes to e-book readers. On a computer, on the other hand . . . I love PDFs! So it really depends on the purpose. Having a variety of formats is good for that reason.

I avoided the Kindle and nook for reasons precisely like that--I wanted lots of options for file formats, not just one or two. For small files, I don't have to convert everything to a specific file format, which is really nice. For the big stuff, I can either use Calibre for a quick convert if I'm in a hurry, or paste into Sigil for the nicer look.

Sigil IS limited in its output format, but it was only designed as an epub editor, because there wasn't anything out there easy enough for newbies yet powerful & featured enough to produce quality files. Since epub is the best format for the Astak, I figure I'll just create that when I do need to convert.

I just loaded the PDF file for "The Zipper" on my pocket Sony Reader (vertical read, screen 3" wide) and it wrapped perfectly with no dangling short sentence ends. I'm guessing the PDF format doesn't have the hard stops of the Word files.
I even increased text size and worked perfectly then too. I was indistinguishable from an epub doc.
So I continue to be happy. If Lauren continues with the demonstrated programs of Filename Z, I'm good to go with that.
regards
Artemis

Sounds like yes, the PDF wasn't formatted with the hard stops, AND your Sony Reader's software has good text reflow ability. Without either one of those two, PDF is a nightmare on e-book readers. Epub, on the other hand, is a standard e-book format designed for e-book readers. Don't get me wrong, I'm glad there's PDF versions too! But not every PDF works well on my e-book reader, so I'm glad I can create epubs.

For my part I'm all in favor of multiple formats. I'm all for making the documents as accessible as possible. I'll be happy to help and when we get going I'll certainly do what I can. I'm happy to have my own works change format. It's the words and ideas, not the delivery mechanism.

I love the 'Z' page. It would be great to have something like that everywhere. However, my suggestion would be to start with all the Kerth stories. If we have to start somewhere, why not there? However, that's just an idea. I'm in favor of whatever we can do.

As for this: WOW! I'm mentioned in the same paragraph as Dr. McKay. Using a hex editor is just a sign that I'm an old-school assembly language programmer. Still... WOW!

Oh, one more thing. In my outline earlier I forgot one important step.

1.5 As soon as you load the file into MS Word, edit the Title and Author in the Properties field. Stanza will pick these up when later when you load the .doc file and you will get your Author and Title exactly as you entered them.

Bob

We don't have consensus on what our procedures will be with epubs going forward, but I've found a file upload script to use for getting epubs on the site. Doranwen, bobbart, it's ready for use if we're ready to start collecting them. Thank you for volunteering your time.

Artemis wrote:

That's great to hear! Hoping other authors will feel the same.

HTML looked to be winning toward the end. It seems inevitable. The stumbling block toward that effort is the sheer number of stories there are to convert. Lots to learn in getting that done. The only place where extra formats are available for stories right now is the Filename Z page. I figured OpenOffice for flexible formatting/printing, PDF for quick printing, and plain text because that's our legacy format. But this is the year of the e-book, so epub is a natural, and a great suggestion from Doranwen!

I'm glad it worked for you, Artemis! I didn't know any e-readers could do flexible word-wrapping with PDF.

Have you made your own PDFs? If not, I highly recommend it. OpenOffice makes it a piece of cake! The PDF files I posted on the Z page were made using OpenOffice (free, open-source, Microsoft Word-compatible, and available here in Windows, Mac and Linux flavors). Just a few simple steps:

1) Use the archive Formatter to convert a story from archive text into fluidly wrapped text.
2) Paste the output into OpenOffice Writer, then clean up the header by restoring the line breaks after title, author name, etc.
3) Do whatever formatting you like. (If you're creating the PDF for printing, you might adjust the font styles and sizes, margins, indentation, format the title as a heading and format the story in columns.)
4) Click File/Export PDF.

I was figuring PDF would be used primarily for printing and tried some different layouts. The PDF for Paul's "Zombies" story is in two columns, magazine-style. I don't know how that one would flow in your e-reader, but two columns seems to be an efficient way to print.

I have visions of filling three-ring binders with fanfic. Would be really nice to have a printer that could print on both sides of a sheet of paper. But Doranwen's print-on-demand idea is even better to dream about. Real, bound books! I'd start clearing shelf space now. The complete and collected works of [insert author name here]. Sigh.

Thanks! Trying to learn programming is maddening.

Darth Michael wrote:

Thanks!

Argh! Sorry about the PTSS. I keep saying to myself, "It should be simple!" But I keep stumbling around in the dark. You sound like a guy with a book of matches. C#/.NET? That stuff looks scary.

bobbart wrote:

Thanks, bobbart. That's a beautiful way to say it.

I think anywhere people would want to start is good.

You're an old-school assembly language programmer? That stuff looks really scary!

Best wishes,

Lauren

P.S. I was working under a bogus assumption about Stanza when I typed this in my previous opus...

The first part is true -- Stanza does strip the extra line breaks from archive format text files, but only for any epubs you save. I assumed that would also hold true for the Word and PDF files you produce using it. It doesn't. Drat. PDFs and Word files saved from Stanza keep those line breaks intact.

But hey, there's still the archive Formatter.

Word 2007 on System 7 on my PC has PDF as a choice, so I simple saved it that way after dl a story from the Archive.
Oh, the formatter is awesome, but I did one without using it and that worked fine too.
One thing I did find out that you have to pay attention to the type font when it is in the Word stage. Not changing it made it really tiny to read at the S setting, and L is too large.
BTW, we submit to the archive in text. Could we submit another copy in PDF or Word.docx directly and would that save time? It has open document as well, if that works better.
The authors could submit their own stories and that gives permission as well.
Thanks again for all your work over the years.
regards
Artemis

Well, I've already started converting the stories I do have saved offline into epub (though going back to the Archive to pull a fresh copy to work from). Sigil is not removing the hard returns so I'm dumping them into Word and doing a series of Replace Alls. Works pretty well, actually.

The one part I'm trying to figure out is just where to draw the chapter line for stories which are not on the MBs (where the Fanfic TOCs are great for seeing the section divisions the author wanted--I've got a tab here in Firefox open right now for doing those searches). I'm tackling Anybody's Baby by Annie M right now, for instance, and finding it a challenge to pick a scene to split at. She's got frequent scene breaks, so any one of them could work, but about how long should a chapter be? (Keeping in mind I'm now looking at a full-screen fic with no premature line breaks, approximately how many pages would you say?) Try to end it on a cliffhanger or suspenseful moment when possible? Might be good to get some guidelines discussed so we have an idea of how to go about it.

As for permissions, considering they've already given permission for it to go on the Archive, I can't see that a different file format would be any issue. We're not adding anything (except for "Chapter 1", "Chapter 2" as necessary) and simply re-formatting. Sounds like we're probably good to go there. So as soon as you've got things set up to do uploading of files, I'm ready to share some. A wiki sounds like a good idea, as long as it's tied to being registered here and not just public, of course. *g*

Are you considering expanding to all four formats at one time? Would you need people to do the converting manually or is there an easy way on your end to batch process the whole lot? (Or perhaps, once there's an epub, it's easier to go from that to another? I don't know of any simpler way to get rid of the hard returns than all the Replace Alls, and I have to think about what I'm doing. For instance, some fics have a space at the end of each line, some don't, and some have sections with three returns where I only want two, in addition to two returns where I only want one--epub spaces out paragraphs naturally--so I have to do at least five conversions to get everything to come out right and be mentally processing how to do them. I'm not sure there's any way to automate that!) Though it looks like the hard returns aren't really a problem with html--they do look a little odd for pdfs, though.

I agree. I don't think there's any reason for authors to object. But if anyone does, it's a simple matter to email or PM me and we'll exempt the story from the additional formats. Shouldn't be an issue, I don't think.

I like the Z page, Lauren!

It would be nice if HTML could be a format choice, too. I recall the last time it was discussed we couldn't reach a concensus on which fonts/colours to use which would enable everyone to view the story. But if it was simply another choice of format, among others, rather than the only choice, then I think that issue would be less important.

LabRat

Well, I am definitely stuck when it comes to deciding chapter breaks. Obviously if it were posted on the mbs, it's simple; just see where the author broke up the parts, and do the same, even if you have to insert a "Part Four" where there isn't a scene break. *g* That way we can be said to be following their very wishes in that respect.

But what do we do with the stories that *weren't* posted on the mbs anywhere? About what size would you break it into two parts? As soon as it hits 30 kb? 40 kb? Or wait till larger? (Obviously, a fic of 200 kb, for instance, needs to be split in places--the question is simply how many and where.) About how long should a chapter be? And if there are frequent scene breaks, what sort of break should we choose to end a chapter on?

If I should be starting a new thread for this, let me know. But I think this needs to be answered so we can do the best job possible. Authors, what guidelines would you prefer if you hadn't posted it on the mbs at all? Anyone have any thoughts on this?

Did you already check zoomway's board? It's got the older stories (dating back to spring 2001) before lcficmbs went online in 2003.

Michael

Ooh, I had forgotten about that! Thanks! That does indeed help--sure enough, Anybody's Baby is indexed there. That should help with most of them.

However, I know there are a few fics that go straight to the Archive, bypassing the MBs--there's one author who does that a lot, I think (the name is niggling at the back of my mind, but I'm having a hard time remembering). What should we do in those cases?

I try to make my chapters about 20K. Some are 30K. Yes, you try to build to a cliffhangar. One thing that will be different on the Archive is that authors often include a Previously section at the beginning of a chapter to bring the readers up to date. That should go away when you are reading a story as a whole. I edit them out when I submit to the Archive.
Good luck with your efforts. For those of us who can change them ourselves, let us know where you want them submitted. Would the Archive be up and ready to do that?
regards
Artemis

Artemis wrote:

I didn't know Word could do that. Cool!

That's an interesting idea. It's more in LabRat's territory, as well as that of the general editors. I don't know if they all use Microsoft Word, or what version. Or if they use OpenOffice , which is totally free and Microsoft-compatible. Terrific program. Had to get the plug in.

You're welcome, and thank YOU for writing stories and contributing to the archive.

LabRat wrote:

Excellent! Thanks, LabRat.

Working on it, Chief.

Doranwen wrote:

Good points, and we've got LabRat's OK. So we're good to go!

I've got an uploader ready and have installed DocuWiki on the archive. (I beat my head against the wall for a while trying to install MediaWiki, but couldn't get it past the database configuration on the archive's server.)

I'll send the links to you and bobbart first for testing. If you see problems, please let me know!

If the wiki turns out to be useful for this project, maybe we can use it to collaboratively work on other projects as well.

It's something I've been considering for a long time, except for the epub flavor -- that's new. I'm hoping to plod away some more at the batch conversion idea, eventually tying it all up with the database. (Batch conversion with the exception of epubs, pointing to the ones you guys will be creating using a new field in our future database/current spreadsheet.) Though people stepping in to help correct formatting post-conversion on the HTML/OpenOffice versions would be wonderful. But it'll be a while reaching that point.

Many moons ago I prepared an archive-to-Word how-to addressing conversion and spacing issues for the archive staff, and it's still online . Recording the steps as a macro turned the whole multi-part search/replace process into a single click. The macro is still there too , but I don't know if it'll work in the current version of Word.

You'll have better results recording all the search/replace steps you go through in your newer version of Word to create a macro more specific to your needs. Things have probably changed a lot in Word in nine years -- and I think I was even using Word 97. Just noticed mine doesn't address turning three returns into two after getting rid of the spaces before and after returns. (^p^p^p to ^p^p, repeat.) Drat.

Anyway, if you haven't, please give macros a try! Could save you tons of time and greatly streamline the process of making a story epub-ready. And if you find it does simplify the job, please consider sharing the macro.

I think it's exciting that we're doing this, and doing this as a group project.

Best wishes,

Lauren

Sending story docs between myself and the GEs, we use text - simply because it seems to throw up less compatibility issues between pcs.

But I think the GEs themselves use various formats when working with the authors. Whatever they're most comfortable with. I don't think they all use MS WORD, no.

I have no idea what would be involved in using something other than WORD or text docs, but that's no reason not to give it a go if it makes things easier for you guys.

Yes, and hasn't that saved my butt time and again over the years. Well, hours of tedious work, at least. <wistful sigh> I really miss that little guy.

LabRat

I know there are a few sites that can change the font/background colors with a drop-down menu (fanfiction.net has recently done this with a dark/light option). It's a fairly simple javascript that can be added to the HTML pages.

I don't know if space is an issue on the archive, but I was recently looking at how to make PDFs on the fly. ezPDF was one option, but I was having problems getting it formatted. The drawback is that every file is formatted the same way (single column, double column, etc). It can pull the format out of the original file, though, so if it's converting an HTML file, it can convert all of the bolds and italics.

I've considering making a macro for it (I have Word 2003 so not sure how easy those are to work with there), but the more I think about it, the more I'm not sure that would work. For instance, some stories have a space at the end of each hard line break, others will need it added when the line break is omitted; those would have to be dealt with separately, and you have to look at the story to tell which is which. Also, I've taken to adding my placeholder symbols at the beginning of the last three lines of the heading info just to save on the hassle of splitting the line later when my Replace Alls remove the line breaks, lol. It's actually quicker to add them beforehand than to go hunt down where to hit Enter later. Some stories have three ^p, others don't. And there's still the skimming through to make sure all formatting worked great--I've found some extra carriage returns at scene breaks here and there (and some scene breaks missing at least one carriage return, causing some bunching up). I do that as I break it up into sections, so it's not really extra work, but does require some monitoring.

So although I'm sure someone macro-savvy could work past those, I've got it down pretty fast so it's not too much of a pain. Three authors done and counting now . . . the only slowdown will be adding them to the wiki. *g*

Thanks for doing the work to get it set up for us to upload, Lauren! I'm glad to be able to share what I've gotten done with the rest of the FoLCs. Couldn't have imagined this could happen, the first time I started reading L&C fic!

Hey! I got one uploaded!

I hope I can be forgiven a little for loading SkyFall first. I wanted to see how a large multi-chapter story would work.

I'll try to get a few more stories done before the evening is out. (And they won't all be my own stories. )

The process to upload seemed pretty straightforward. I have many more stories in epub format but I want to go slow and make sure I've done the best job possible to get the format clean.

Thanks again, Lauren. And you too Doranwen. I'd like to go faster but time is a precious commodity these days. (Not to mention a WIP that is demanding my time.)

Bob

Hehe, no worries about starting with your own stories first. I don't have any of my own (not in L&C anyway--I *have* written fanfic for early Smallville, for X-Men movieverse, and most recently Narnia, but never felt like I *got* L&C well enough mentally to write them). And my motives are purely selfish, I must admit; I want to read my favorite fanfics on the Astak!

I also have been slowing down a bit now to re-read as I format. It's too tempting. *g* (It's also good for the story--I'm more likely to catch formatting goofs, and once in a while misspellings--I hope it's OK if I fix things like "jOKing", and add in missing words e.g. "asked of [her]". Only the obvious mistakes, though. I'm OCD enough that I can't stand leaving a misspelling in, even when trying to replicate material precisely.)

I'm holding off on uploading more until I figure out what to do about the filenames. I've been using the ones the Archive has (goofed on one of them, but I begged Lauren to fix for me ), until it occurred to me that the Archive seems to sort them out by year, which means theoretically some stories have identical filenames, they're just in different folders. In the interests of saving her any MORE editing work (can't have duplicate names!), I figure I'd better get that sorted out before uploading a ton. But I do have a good deal of files converted: all the stories from Annie M, Aria, and Becky Bain (I'm still finishing Timeless), so those should appear in a bit. Next up looks like Carol M. (My offline archiving is based entirely on the first letter, since so many nicks are not first & last both; hence Nan comes under N and Wendy comes under W--so while I'm uploading and adding them as the Archive has them, the order I start on them is a bit different.)

Doranwen, could you mail me the process you are using to convert the files. My first two attempts don't look nearly as nice as yours.

As for names, I was using the same ones as the download. I figured that would prevent duplicates. If we need to have some other file name rules, then we all need to be on the same page for that. Earlier tonight I established that if you upload a file with the name of a file that is already there, it replaces the old file and all the links will point to the new one. So duplicate names are a BAD thing.

If file names are unique by year, we could use the same name preceded by the year. For example, hello.txt would become 1996_hello.epub. But some files don't have the year in them.

Anyway, just so there is a rule.

Bob

I'll start an e-mail right away.

As for the filename stuff, I'm waiting to hear back from Lauren on that. I figured I'd have her make a final decision on how things should be named. But yeah, I was assuming I'd use the Archive's names as-is, and the ones I uploaded all use that. (I forgot to change the very first one, so uploaded it with my naming system, and had to ask Lauren to fix it. )

*Finally* finished with Timeless! I had to really slow down because I couldn't help re-reading the whole thing as I broke it into chapters (39, the record so far--I tried to make them somewhat smaller chapters than some previous--I'm vacillating between 4-9 pages on this one, 7-13 on Anybody's Baby--kb amounts don't help so much when I'm trying to separate big fics so I'm trying to gauge about how many pages should roughly be in a single chapter and go off that). I know nothing of Beauty and the Beast (other than vague images from the Disney animated movie that I glimpsed briefly as a child), but she manages to make me interested in it as well as glued to the screen as I read. Simply amazing. And as soon as I know what name to give it, I'll have that (and others) in epub up there for y'all to enjoy.

Doranwen wrote:

Fear not! All story filenames in the archive are unique. I make a pass through the filename catalogs every week to make sure each new story filename is not already in use. (If anyone notices a filename that is used more than once -- which would most likely be the result of me messing up placing a story in alphabetical order in the filename catalogs -- please let me know!)

bobbart wrote:

Sounds perfectly acceptable to me. Your table of contents page looks great.

Understood. We're grateful for any you and Doranwen convert.

bobbart wrote:

I thought "When the Sky Falls" looked good, bobbart. It had the Stanza-type intro, though, which places the title by itself on the first page, with a few blank pages interspersed here and there. That happens to me too making epubs with Stanza. Not onerous. The epubs I tried with Sigil didn't do that, though. They looked a little more polished, more like Doranwen's, whose epubs are amazing.

All you FoLCs hooked on your ebook readers are in for a treat! I don't see any reason why we can't open it up soon. Doranwen and bobbart rock!

Lauren

WOOHOO!

Just wait - OTOH is way more than 39 chapters :p . Though it's already broken down for you at least...

I have Stanza on my iPhone and am looking forward to being able to read fic on there.

As a moment of pondering though...

I tried to upload OTOH [just did a 'save as epub' on it, nothing else] and I had a hard time getting it to load and eventually loaded it in three smaller files. Does anyone else have any problems with large files? While it is the largest, it certainly isn't the only one that might have issues... There's several others on the 'really really long' list...

Carol

Kinda like Masques? *g* I know there are others longer now, but that one was THE long one for so long it always comes to mind. (You're immortalized in memory, Labby. *g*)

As for getting files to load faster, splitting the internal XHTML sections is the key. Many conversion programs can be told to automatically split it every so many kb, but you want to split at chapter sections, really. Short chapters will load much more quickly. (And they mean short in terms of book length chapters, which is longer even than I've been splitting here.) So as I'm converting the epubs, I split at every chapter so each chapter is a separate XHTML file. The reader won't see it to know it's not all one file--the epub structure is *designed* as a zip containing the XHTML files, images, and even embedded fonts (I don't think we need to mess with those for this purpose, lol)--but it'll load faster than if it were automatically converted to an epub with a single large XHTML file inside.

So anything I create *should* load fairly quickly (though sheer file size will have an effect--a 2 mb epub will take longer to load than a 20 kb one), but let me know if you have problems with it.

(And I've gotten started on your fics now, Carol. Glad you're excited! I'll upload them all when I finish them.)

LabRat wrote:

Sorry for the delay in getting back to you on this -- it's a very interesting idea.

It probably would save a few steps in epub creation if we standardized on a word processor format, but I think the larger picture is how you, the general editors and the writers interact. We definitely don't want to throw a monkey wrench into a system that works. That said, if you all are looking for a change and are interested in moving away from the text-only format, that's something we can explore.

If you want, LabRat, you could start sending me the weekly upload stories in whatever format you receive them in from your general editors. Or if you do some editing on them after, you could send them on to me in Word. I can convert copies to text for posting on the archive and keep the originals on the back burner for conversion experiments.

It'd be nice to standardize on something that preserves basic formatting (bold and italics), that eliminates the need to worry about line breaks, and that poses no compatibility problems.

Actually I think MS Word would be a good choice if we were to standardize on a file format for story submission. Not Word itself, necessarily, because Word is expensive, but the Word .doc format. I think it'd be a bad idea if people felt they had to buy a copy of Word in order to write fanfic. Fortunately lots of word processors these days can read and write Word .doc format -- even the free ones.

I'm curious about the compatibility issues you mentioned, especially involving problems with story files in different flavors of Word. Is that a problem? Files saved in Word 2010 vs. Word 2003 vs. Word 97, etc.?

I definitely do not want to make your job as editor-in-chief more difficult, LabRat. Or that of the general editors.

Would changing things make it easier? And are there program features not being used now but which could improve the experience for writers and editors? Redlining/revision tracking, for example?

Any writers or general editors, if you're reading this, have you experienced any compatibility issues involving your word processor of choice when you've sent or received story files? Is there something we could do or decide upon that would make your job easier? What are your thoughts?

Speaking of free word processors that can read and write Word .doc format (from a few paragraphs back), just to throw out some options...

* OpenOffice can read/write Word docs in a variety of flavors (Word 2003, Word 97/2000). It's not a Word clone, it's a suite -- an MS Office clone. Well, not a clone exactly, but very much a work-alike. Available for Windows, Macintosh and Linux. OpenOffice Writer can save documents as PDFs without requiring Adobe's expensive PDF creator. Its native format is an open document format that allows programmers to easily get at the file's innards.

* AbiWord can also read and write Word files. Available for Windows and Linux. AbiWord is small and fast and speedy even on older PCs. The download is under 10Mb. It feels more responsive to me than OpenOffice Writer.

* Google Docs is for writing in the cloud. If you've got a Gmail account, you've got Google Docs. It can "upload" and "download" files in MS Word and OpenOffice formats. You edit and save your Google doc files online, so they're out of harm's way in case of hard drive crashes. Google Docs also allows collaboration -- up to 50 people can work on a document at the same time! Can you imagine? I wonder what it'd be like to use it to write a story with several other people in real time. Chaos? Genius? Any authors out there willing to put their heads together and give it a try -- and then report back on the experience?

Other options I've read about but haven't tried: ZoHo Writer is part of a large online suite of tools and also allows collaboration. Microsoft's Office Live is supposed to allow online collaboration, but I'm not sure if you need to have purchased a copy of Office to use it. I've heard some buzz about other Microsoft tools in the cloud too.

My personal preference for my own word processing is either AbiWord or OpenOffice Writer. I know there are some quirky things about OpenOffice writer, and that it's a little clunky compared to Word, but it does most everything I need it to do. Except let you find and manipulate paragraph markers, like Word can with its ^p. (AbiWord can't do that either.) But I've found other ways. The text editor Notepad++ -- also free -- is great at searching/replacing returns and tabs. (Instead of Word's ^p and ^t, it's \n and \t)

My favorite text editor is an old program, Arachnophilia. The maker stopped programming for Windows and does a Java version now, but the interface changed so drastically I stayed with my old tried-and-true. You don't hear much about Arachnophilia 4.0 these days, but it's still my go-to tool. It deals effortlessly with paragraph marker and tab search/replace operations. It's not freeware, though, it's careware. You have to be a nice, non-whiny person for a day to use it.

I've always been interested in free and open-source software, but ever since my laptop was stolen a few years ago, I've been almost obsessed with it. Much cheaper to replace software that didn't cost anything in the first place. I lost Quark XPress when my laptop was stolen. It was an older version I registered off a British computer magazine cover disc, so it was sort of free. But it had a one-time registration code and I wasn't able to re-register it. Buying Quark would've set me back $800+. Anybody know of a good free/open-source desktop publishing program?

LabRat wrote:

I'm sorry the macro didn't work for the newer Word. Don't you love it when software makers "improve" their program, breaking something you liked about the old version? That happened to me with Paint Shop Pro. Version 7 -- ugh. Version 6 -- sweet spot.

Karen wrote:

Definitely something to add to the to-do list! Thanks for the suggestion, Karen. For long streches of reading, I prefer reading light text on a dark background and use EditPad Lite for reading stories offline. It lets you configure font styles, sizes, and foreground and background colors. Something like that would be very cool on the archive, and very restful for the eyes.

Speaking of space issues, when I logged into our control panel the other day, and it's been a while since I'd logged in, one important piece of information was missing -- the part where it shows you how much space the archive is allotted and how much is left to use! The contract with our original Web host allowed 250Mb for storage with unlimited bandwidth, and we'd been inching closer to the limit. I called to find out what was going on, and it turns out we've been put on a new plan -- unlimited space, unlimited bandwidth. So, no more nail-chewing over space. We've got room for multiple story formats.

Thanks for the tip on ezPDF too. It sounds like it does exactly what we'd need. It might not be a problem for us if every file is formatted the same way.

Some epub news: Thanks to Doranwen and bobbart, there are 50+ epubs now on the archive. We'll open it up soon. Watch for an announcement probably sometime this weekend.

Best wishes,

Lauren

Wow, great to see this progress

A couple of words on formats, etc.

First off, I’m pleasantly surprised that Word’s classic doc-format is supported by more than Open Office. I figured we would need to use RTF instead to make the Archive-upload open to anybody. Of course, there’s always the fallback-to-txt option in case of a non-standard word processor. As for the different versions of Word, when you select the doc-format, it doesn’t matter whether you’re using Word 2003, 2007, or 2010.

Change tracking is a feature I’ve used with all stories I submitted to the archive so far. Don’t know how much this has actually become a defacto standard between GEs and writers, though. I’m sure Labby could make a poll with his GEs

As for the original upload in a non-ASCII-txt format and the subsequent conversion to ASCII-txt. I’m all for doing it this way, but I believe there is one style-issue we should be aware of. Right now, the Archive presents all stories in plain-text ASCII. Well, duh, because that’s what this thread is all about. But there’s the matter of the typographic characters such as slanted quotes, the ellipses character (which is a stylistically/grammatically suspect entity in and of itself), and the n-dash and m-dash which look indistinguishable in plaintext. In a lot of stories, those characters get replaced by standard-ASCII characters, e.g. using two or three regular hyphens instead of an m-dash. Whether it’s the authors doing this or our dedicated GEs, once we switch to uploading the stories in a richer format, I believe we should standardize those conversions.

Michael

I've some experience with Word compatibilities. Those of us with System 7 Word 2007 (.docx) must save to Word 2003 (a compatibility feature available easily). Unless you have Word 2007, you cannot read .docx. My recommendation would be using Word 2003 as a standard (Windows XP).
When I submitted to the Archive in txt, my GE and I communicated changes in Word 2003. txt is just too hard to read and correct. Then she converted the changes back to txt for the Archive. Eliminating all that would be easier for Word users.
Keep up the good dialog.
Artemis

I've been converting stories (Just a few. Doranwen has done most of the conversion work.) and I have had a change in opinion about splitting long stories.

When I started, I was thinking that the epub conversion editor should "bend over backwards" to put the breaks where the author wanted them. If there weren't chapter breaks in place already, we should go back to the original posts and/or contact the author.

That was how I started. Over the past few days I've realized that would make the process too onerous and will impede the epub conversion process.

My epiphany was that if an author of a long work felt strongly about breakpoints, they would have put in chapter or section markers in the first place. I feel that way, which is why all my longer works are chapterized in the archive.

So, I think that for epub conversion, the rules should be simple.
1. If chapter/part breaks are in place, honor them even if they are small. (Take a look at Meet Me in Kansas City.)
2. If there are no preexisting break points, put them wherever they seem to fit with minimal disruption to the story. If the author (or anyone else for that matter) doesn't like the break points, they can redo the conversion themselves or at least request a reformat. For my part, if I did a conversion and the author wanted it segmented differently, I would be happy to make the change ASAP.

Just some thoughts after a few days of converting stories.

Bob

I do think we should look at the TOCs to see if there is one for the story in question, and if so, follow it--but contacting the author beyond that I'm beginning to think ought to be completely optional.

So maybe we make a general announcement that if an author wants a very old fic (many of which had stuff hosted on the old mbs and the actual story parts are gone so impossible to tell where the breakpoints were) split a certain way, they need to dump it into a doc file and split it and then offer to e-mail it to whoever's converting the fic . . . How does that sound? Put the onus of that on authors, because frankly, with the sheer amount of fics I'm converting, I don't have the time to contact a half dozen authors about two or three stories apiece.

Doranwen, you said: My point is that I don't think that's necessary.

My epiphany, such as it was, was NOT that we can ignore the authors wishes. My belief is that the lack of defined breakpoints in the archive is a reflection that the author didn't feel that particular breakpoints were crucial to the story. If they believed that they were, there would be chapter markers in the archive.

I'm not trying to ignore the author's wishes. However, for me, time is something I always have to factor in to an activity. I found this weekend that it was taking me *longer* to try to find those original posts than all of the rest of the time for the epub conversion put together.

So, if I believe I'm going to have to go on a search for the original posts for every long fic I feel inclined to convert, then I'll be leaving this effort to someone else.

Bob

Sentence breaks, paragraph breaks and scene breaks are all matters of grammar. Chapter breaks are a matter of choice. Terry Pratchett only has chapters in his YA books. The main Discworld novels have no chapters. So if the author hasn't defined them the editor shouldn't insert them.

I completely understand. However, if there are not defined breakpoints, there are two problems:
1. Some e-readers - Stanza for example - will put them in anyway. I can make a better breakpoint decision than Stanza.
2. When a document/story/book is large, the e-reader can get sluggish in handling the stories that are not segmented.

I guess it's obvious I got worn down this week with this effort. I believe in it, but at some point it's not worth the hassle. I'll continue to convert stories for my own use. I WILL add breakpoints in long stories (again for my own convenience) and if no one likes my decisions, I simply won't share my work.

Grumpy Bob
(Believe it or not, that is the name on my cubical marker at work. I get ... testy. And when I was younger, I did sometimes take my ball and go home. I better shut up before I get *everyone* mad at me.)

This one thread seems to have become a very large rock rolling down a steep hill. It sounds like you two are proposing to convert the whole Archive in one fell swoop. How about letting the authors who are willing convert their own stories if the process is simple enough? In that way, you are also guaranteed the author's approval about the new format.
Granted there are old stories with the authors no longer active, but two or three people don't need to do all the work. Shouldn't the word be spread wider?
Also you talk of "uploading" it now. My understanding was there was a filter between author and Archive. Where and what is the link?

Artemis

Lauren's been checking over what we're up to, and had us put together a list of the steps we follow (involving Sigil), so anyone else who wants to can convert their stories to as high-quality epubs. She'll be posting that sometime soon so any author can convert their own if they like and have them look really good. So don't worry, I gather that pretty soon it will be thrown open to everyone. She just wanted to make sure they could turn out really nice and the system works fine first.

I don't want to jump the gun on her, so I won't post any links, but she's got a separate system set up where Bob and I have been uploading the finished epubs and putting together a wiki of the ones we've done so anyone could get them. I'm gathering she'll have that released within the next day, so all you FoLCs can go grab epubs to read. Ultimately she'll merge it with the Archive, I think, like the Filename Z page is.

As for the TOC, Bob, I don't think it's that hard to do a quick search. I have two links open--one for the mbs, and one for the old boards. If I hit any trouble finding it there, I just split myself. But I actually find it easier not having to think about where to split. The author DID, in some ways, tell where things should split when they broke it up in sections to post. The ones who never posted to the mbs at all, that's pretty clear they didn't care about splitting, so we can do it where we think best. That said, it's probably not too much of a big deal to split it where you want if you find checking the mbs is too time-consuming (so don't let that keep you from converting some)--but I wouldn't do that with authors who are very much active and might convert their stories themselves anyway.

The big reasons for splitting are twofold, actually. One, on an e-book reader, you can't just drag the scroll bar halfway down the screen to read the last half. Dozens of page turns are time- and battery-consuming (if my experience with my e-book reader is anything like the norm), and anything beyond a few page turns is a pain to get to. Epubs have a built-in TOC system designed to make jumping around more convenient, but you have to mark something with a heading system to get it to work (Heading 1, Heading 2, Heading 3, etc. like in Word). (Unless you're a guru at the xhtml editing, which I am not--it's a lot of coding to do it without the heading stuff.) The obvious thing to mark are chapters or sections, whereupon people can then go to the TOC and pick Chapter 4, or Part Six, as they like.

The other reason for splitting is the physical splitting of xhtml files. It's not a good idea with an epub of any larger than about 200-some to leave as one big xhtml file. You can mark chapter headings within that, though--you can mark chapter headings without splitting physically, but you wouldn't want to split physically without marking chapter headings because physical splitting means it starts on a new page and random jumps to a new page look tacky. The splitting prevents the file from loading sluggishly, or worse, freezing the reader. I have no doubt that an un-split Masques, for instance (or On the Other Hand, which as an epub came out to 948 kb, I found!), would freeze a good number of e-book readers. It's simply a size beyond what they're designed to load into memory, and that's what the e-book reader has to do with each physical xhtml file. Splitting it means it only loads the one xhtml file the reader's on, then when they hop to the next file, it loads that one instead. It'll look like one big file to the reader, just with page jumps, but the difference in responsiveness is huge.

Okay, let me try once more and see if I can manage not to appear to be trying to create problems.

I do almost all of my reading on my iPhone using Stanza. To facilitate this, over the past few years I have converted almost 200 LnC stories to epub format.

When this discussion got started, I thought it would be a chance to share the work I've done. Unfortunately, when I started considering the idea of posting the epub versions of the stories I've converted, I quickly realized that the formatting I had used was too crude to offer to anyone else. I respect the authors and I feel it is being disrespectful to share a version of their work where the format is less than the best I can make it.

So, with a LOT of help from Doranwen and Lauren I've been converting stories. Using Doranwen's approach produces a *very* good looking work.

As far as segmenting goes, it shouldn't be a problem but it is. Anyone that has loaded a large story into Stanza will understand. I have been segmenting all the stories I've converted for a long time.

Believe me when I say that it is easier not to segment. If I skip the segmenting step I can convert almost any sized story in between 5 and 10 minutes. Adding segments will often double that time. Segmenting takes a lot of work. However, it's necessary to have an end-product that is clean enough to honor the authors whose work we so enjoy.

My point earlier is that the segmenting process is not changing the story. When you print a story on paper (which I have also done) you are artificially introducing segments -- pages -- that have nothing to do with what the author intended. To me, the segmenting that we are talking about is nothing more than making sure that when the e-reader handles the story, it neither chokes nor puts a section-break in a place that disrupts the story.

Back in hiding now...
Bob

Thanks for the explanation, Doranwen & Bob. That makes sense. It just seemed confusing without that inside knowledge. Actually, I don't have time to do any converting of my stories now. Maybe later. So if one comes up alphabetically, you're welcome to do it.
Artemis

Artemis wrote:

It will be, Artemis. I think Doranwen and Bob are doing a great job at the starting line, but the race will be a long one. Should be fun too. Hopefully more people in running shoes will show up.

Since Doranwen and Bob both said they had been creating and stockpiling epubs, it seemed logical to ask if they'd be willing to test out the cobbled-together uploader/wiki system and hopefully target trouble spots before we opened it up. For anyone who's been eager to get at those epubs to read, or who's interested in submitting some, I'm sorry for holding out on you. I wanted to make sure the system was workable first.

I've been cheering on Doranwen and Bob -- and it's amazing how quickly the story count is climbing. Artemis has a good point, though, you guys. Please don't burn out. You've got us off to a great start. There's enough there now to keep people happily buried in their ebooks for weeks!

There's an uploader tool that can be used to send stories up to the folder where the epubs are stored. Bob pointed out a flaw, and I'm unsure of its security and need to find an alternative. I'll publicize the link once a safer tool is in place. Shouldn't take long.

How about this: Anyone else who wants to create and submit epubs in the meantime, just e-mail them to me as an attachment at LCFanfic [at] gmail.com

Yes -- an editor-in-chief and her team of general editors, who do a great job!

The situation with submitting an epub is a little different from that of sending in a new story. The epub has already been through the editing process. It's the same words, just different packaging. So I thought I'd expedite things a bit with a direct upload tool, and now I'll be tweaking that part. And by using a wiki page to hold the links, we can collaborate, and I won't be the bottleneck. Needed to do a bit of testing first.

Doranwen wrote:

Exactly! Merging it with the rest of the archive will take a while. I hope people like the wiki approach for now.

Best wishes,

Lauren

Attention, fans of e-books and Lois & Clark fanfic -- 80+ stories are now available to read in epub format. Get them at the archive on our new epub wiki page .


The whole archive isn't represented there yet, but there's enough to keep you busy for a while. Doranwen has been working on an alphabetical approach by author, and Bob Bartholomew has contributed epubs as well. I'm going to post some stories I've been meaning to re-read. And you can become an epub editor too by converting and submitting some epubs!

The idea is that epub editors -- i.e., you -- can upload epubs and create the links readers will use to download and read them. That'll be the plan once the uploader tool is tweaked. For now, if you've got epubs you'd like to submit, please send them to me as an attachment at this address: LCFanfic [at] gmail.com.

I'm not finished working on my part of the "intro to epubs" wiki page, but you know what? Wikis are works in progress. They're ever-changing. And they're all about collaboration -- so if you spot something wrong or that needs elaboration, you can log in and fix it.

And if you're wondering why the wiki pages don't look like the rest of the archive, it's because I haven't figured out the skinning yet. Works in progress, right?

Please give a hand to Doranwen and Bob Bartholomew!

Happy reading,

Lauren

I was excited to find you had done my "Lois Lane, Last Will and Testament" story and I loaded onto my reader. It worked great. Just a small comment though. Your story titles are humongous compared to the size of the text. I have to read at the M setting because the type at S (which is where it starts and loads best) is too small for me to read.
Are we constrained to use the type on the Archive?
And what is that size? Courier 10? Even the Archive is too small for me to read and that is one of the reasons I put the stories into Word and use Arial 12.
regards
Artemis

Yeah, the story titles are meant to really be on a title page of a book, I think (but since we don't do title pages with fanfics, and I didn't want to change things THAT much, it's a bit out of place). On a computer screen it looks fairly nice, but I suspect it's big on an e-book reader. *g* Haven't figured out exactly how else I'd like to do it--heading size is hierarchical, so one has to be careful or the TOC will look rather odd.

The font size for ordinary text, I think is a default thing. I don't think I have any control over it, other than to mark things with heading numbers (which then enters them into the TOC--not something you want for an ordinary paragraph!). So I guess you'll have to keep flipping them to the medium size. My e-book reader has 9 levels of zoom with epubs, lol, and usually the 2nd level (one zoom click) is very comfortable for my eyes. Enjoy the fic!

As of a moment ago we have 96 stories converted and available on the epub site. I'm sure we'll pass 100 before tomorrow morning.

I have noticed one minor but irritating issue with these conversions when I'm using Stanza on my iPhone. When I load a story, everything seems to be correct except that Stanza is not picking up the title properly. Every story that I've tried comes across with the title shown as 's' in the ebook directory. Once you open the document, the title is correct, but I've had to fix the outside title by manually changing it after loading. Strangely enough the author comes through correctly.

I would ask that others try to download some of these and let us know if you see this or any other problems with these conversions. If we need to fix anything, it will be much easier now than when there are 500, 1000 or more that have been converted.

Bob

Congratulations, Bob! I just dl "Hottest Team in Town" and the title was fine in the list. I have a Pocket Sony Reader. So no problems here.
regards
Artemis

Just tried the add-on for Firefox called EPUBReader and it works great! For anyone who'd like to read the epubs in-browser (and have the nice formatting and chapter navigation and all), once you install it, you just click on an epub link and it loads in a new tab ready to read.

I added the link to the wiki page all about epubs so anyone can grab it.

With Sony Reader on my computer, the epub came right up ready to read the minute I dl it. No need for the Firefox add on. I do have Firefox, though and it's a great idea.
Thanks for all your hard work.
regards
Artemis

Is it possible to add the summaries to the wiki pages?
I find archives which tell me nothing about to expect from a story pretty useless and I can't remember all of them just by title and author from the main archives.

Well, I think the point is to eventually merge the two (add the epubs from the wiki to the Archive), and having all the summaries would make adding the links on the wiki more difficult for those of us contributing. I recommend simply having two tabs open--one of the Archive, one of the wiki (whether you're doing it by author or by title doesn't matter since you can choose either one in both locations)--and go through the fics that way. Then you can read the summaries on the Archive and grab the matching epubs you want.

Just had an interesting thought come up, and was discussing with Bob; it occurred to me to have some other people test things in actual e-book readers (this excludes Stanza Desktop, as well as the Firefox add-in).

If you have an e-book reader of some sort that handles epubs natively (Nook or iPad or whatever else out there; Kindle being the big exception), would you download a copy of Nan's fic Teamwork as epub and test it out? I'd like to hear what the TOC navigation experience is like. Can you hit the top of the fic by going to the title, above all the chapters, or does it only let you go to the second layer of the TOC and just choose a chapter to go to? I only have the Astak, you see, which is very limited in TOC navigation, and I'm wondering if the others are as limited, or whether they have other options.

(The Astak, to note, is not a touchscreen device--it uses E-ink--and only has the numbers 0-9 as input options, plus a select and a quit button. It also is limited to 8 options per screen before having to "turn the page", meaning more than eight chapters in a fic, and one has to flip the page just to see the ninth chapter to select . . .)

If I read the doc correctly adding a file called in the wiki conf directory and putting

in that file will let you open the file in Stanza directly on a mobile device like my iPod Touch. I can currently get to the epubs but what I get is the raw file shown in a Stanza browser window.

Not having a doku wiki to experiment on I can't be sure that works.

Doku wiki Mime doc
Stanza developer doc Near the bottom under Links in Book Descriptions.

Hmm, no clue how that would work. There's got to be an easy way--usually you sync a device with a computer library. Calibre appears to do that rather well. (I create my own directory structure on my e-book reader so I don't use it much, but it will manage it all for you if you don't want to handle it, and it supports a ton of different devices.)

Sorry, got behind in this thread.

Michael wrote:

That'd be cool! It'd take time -- and lots of procedural changes. LabRat and her General Editors would need to be heavily involved.

AbiWord and Google Docs do an impressive job of reading Word files. Do they support Word's change tracking? Not a clue. Google Docs has its own version of it though.

I didn't know that. At work we use 2003, and that's the latest I've used.

Excellent point. Not sure how we'd implement, but we'd certainly want to be consistent.

Artemis wrote:

Interesting. I'd love to poll the General Editors to find out what word processors they use to work with authors -- and what they'd be willing to use. And for that matter, what authors currently use and would be willing to use.

How important is change tracking? Does everyone use it?

Dcarson wrote re the wiki conf directory:

I'll give that a try (hope it won't keep the Firefox epubreader plug-in from working). Left-clicking the epub filename in Firefox was bringing up the raw file for me too until I installed the epubreader plug-in Doranwen recommended. Right-clicking worked to download it though. Internet Explorer, however, brings up the download dialog box with no problems.

I'm using Stanza on the iPod Touch too. Are you getting the raw file problem there? How are you pulling down the epubs? These are the steps that work for me:

* In iPod/iPhone Stanza, tap Get Books at the bottom of the library screen.
* Tap the + plus sign at the top of the Get Books screen. (If you've recently downloaded something, you may need to tap "Edit" first.)
* In the input field where it asks for "Valid ePub/eReader URL," finger-type this: lcfanfic.com/epub/cause.epub -- or any "filename.epub" (To find out the story's filename, hover over its epub link in your computer's web browser.)
* Tap the blue Download button in the upper-right corner to bring the story into your library.

That's a lot to go through. I don't know if there's a way to do it by tapping on the link from Safari, but will give the mime thing a go.

Best wishes,

Lauren

I can only talk about my couple of fics, but every time I upload one, my GE's first suggestion was Word+Change Tracking.

To me, it's certainly the tool with the least amount of friction when it comes to editing and I wouldn't want to miss it for the world

Michael

I couldn't track where this came from, but that's not true.
The later ones, 2007 and 2010, can read the earlier ones and load them in a "compatibility mode".
Earlier versions cannot read .docx from 2007 on. So if you want everyone to be able to read the word doc, you must put them in 2003.
I guess I don't really understand why the simple reader, not an editor, needs elaborate document tracking and a detailed TOC. You don't have that in a book. If you forgot something in the plot as a reader, a simple word search is adequate, IMHO.
Hope I'm helping and not confusing the issues.
Artemis
P.S. Having thought a little longer, I see where epub doesn't care what version of word originated it, but Word to Word, my statement holds.

Artemis, that came from me. And I was talking about the doc-format (Word 2003 doc), as in the doc-extension and opposed to docx.

And as far as doc is concerned, once all clients are Word 2003 and above, you don't have troubles with doc. Plus, I believe you get more/better 3rd-party support (Open Office, etc) with doc, since its older. Back with Word-XP, Word-2000, Word-98, the doc-format always meant backward compatibility issues and you had to make sure which version exactly you saved to if you needed to talk to older clients. Now, with doc vs. docx you can simply ensure this via the extension (since Word kindly saves Word-2007 only as docx)

The entire subject of DOC-files only concerned the communication between authors and GEs, not the final product on the Archive, IIRC. Right now that happens in plain-text, again for compatibility reasons between Labby, GE and author. And, unless my impression is off big time, most GE/author communication is happening using doc-format and files that got converted to plain-text, back to doc, and then again to plain-text for uploading.

Hope that clears it up

Michael

Edit:
Oh, and yeah, there's the compatibility pack so Word 2003 supports docx as well, but that means a separate download etc and that's not something you want to require from somebody you want to share a document with

Thanks for that, Michael. Yes, I definitely see the utility of change tracking in the editing process, GE to author and back. I was just confused as to why that would relate to the final product's conversion to epub. So it should be no concern to the people converting someone else's story.
regards
Artemis

I just noticed that the stories submitted in January are all available in multiple formats.

That's cool! And great work

Michael

It is, indeed, very cool. Well done, Lauren!

LabRat

i can read epub files on my nookbook device along with pdf files. The zip epub files won't work on a nookbook.

Hi Shelly:
The epubs are not zipped. Look here for folcs who did older stories: http://www.lcfanfic.com/wiki/doku.php

theres a site under movie fansite where they zip the file

We've passed a milestone with our collection of older stories formatted in epub: more than 400, if you look at them in the "by author" list. Slightly fewer in the "by title" list. The Kindle-formatted side has a way to go to catch up with around 60 titles. (Though if you were lucky enough to get a Kindle Fire for the holidays, you can also take advantage of epub goodness.)

Many of those epub files can be attributed to Iolanthealias, who has done a marathon converting job. Thank you, Iolan.

Shelly, I'm sorry to hear you're having difficulties with the epub files. An epub actually is a form of zip file -- you can even unzip one with any unzip program to see all the separate files that form it. It's my understanding that the zip container is part of the spec.

Is anyone else with a Nook having trouble reading any of our epubs?

Thanks,

Lauren