Lois & Clark Forums Forums Fan Fiction Fanfic Related Fanfic in epub formats - archive anywhere?

Hehe, no worries about starting with your own stories first. I don't have any of my own (not in L&C anyway--I *have* written fanfic for early Smallville, for X-Men movieverse, and most recently Narnia, but never felt like I *got* L&C well enough mentally to write them). And my motives are purely selfish, I must admit; I want to read my favorite fanfics on the Astak!

I also have been slowing down a bit now to re-read as I format. It's too tempting. *g* (It's also good for the story--I'm more likely to catch formatting goofs, and once in a while misspellings--I hope it's OK if I fix things like "jOKing", and add in missing words e.g. "asked of [her]". Only the obvious mistakes, though. I'm OCD enough that I can't stand leaving a misspelling in, even when trying to replicate material precisely.)

I'm holding off on uploading more until I figure out what to do about the filenames. I've been using the ones the Archive has (goofed on one of them, but I begged Lauren to fix for me ), until it occurred to me that the Archive seems to sort them out by year, which means theoretically some stories have identical filenames, they're just in different folders. In the interests of saving her any MORE editing work (can't have duplicate names!), I figure I'd better get that sorted out before uploading a ton. But I do have a good deal of files converted: all the stories from Annie M, Aria, and Becky Bain (I'm still finishing Timeless), so those should appear in a bit. Next up looks like Carol M. (My offline archiving is based entirely on the first letter, since so many nicks are not first & last both; hence Nan comes under N and Wendy comes under W--so while I'm uploading and adding them as the Archive has them, the order I start on them is a bit different.)

Doranwen, could you mail me the process you are using to convert the files. My first two attempts don't look nearly as nice as yours.

As for names, I was using the same ones as the download. I figured that would prevent duplicates. If we need to have some other file name rules, then we all need to be on the same page for that. Earlier tonight I established that if you upload a file with the name of a file that is already there, it replaces the old file and all the links will point to the new one. So duplicate names are a BAD thing.

If file names are unique by year, we could use the same name preceded by the year. For example, hello.txt would become 1996_hello.epub. But some files don't have the year in them.

Anyway, just so there is a rule.

Bob

I'll start an e-mail right away.

As for the filename stuff, I'm waiting to hear back from Lauren on that. I figured I'd have her make a final decision on how things should be named. But yeah, I was assuming I'd use the Archive's names as-is, and the ones I uploaded all use that. (I forgot to change the very first one, so uploaded it with my naming system, and had to ask Lauren to fix it. )

*Finally* finished with Timeless! I had to really slow down because I couldn't help re-reading the whole thing as I broke it into chapters (39, the record so far--I tried to make them somewhat smaller chapters than some previous--I'm vacillating between 4-9 pages on this one, 7-13 on Anybody's Baby--kb amounts don't help so much when I'm trying to separate big fics so I'm trying to gauge about how many pages should roughly be in a single chapter and go off that). I know nothing of Beauty and the Beast (other than vague images from the Disney animated movie that I glimpsed briefly as a child), but she manages to make me interested in it as well as glued to the screen as I read. Simply amazing. And as soon as I know what name to give it, I'll have that (and others) in epub up there for y'all to enjoy.

Doranwen wrote:

Fear not! All story filenames in the archive are unique. I make a pass through the filename catalogs every week to make sure each new story filename is not already in use. (If anyone notices a filename that is used more than once -- which would most likely be the result of me messing up placing a story in alphabetical order in the filename catalogs -- please let me know!)

bobbart wrote:

Sounds perfectly acceptable to me. Your table of contents page looks great.

Understood. We're grateful for any you and Doranwen convert.

bobbart wrote:

I thought "When the Sky Falls" looked good, bobbart. It had the Stanza-type intro, though, which places the title by itself on the first page, with a few blank pages interspersed here and there. That happens to me too making epubs with Stanza. Not onerous. The epubs I tried with Sigil didn't do that, though. They looked a little more polished, more like Doranwen's, whose epubs are amazing.

All you FoLCs hooked on your ebook readers are in for a treat! I don't see any reason why we can't open it up soon. Doranwen and bobbart rock!

Lauren

WOOHOO!

Just wait - OTOH is way more than 39 chapters :p . Though it's already broken down for you at least...

I have Stanza on my iPhone and am looking forward to being able to read fic on there.

As a moment of pondering though...

I tried to upload OTOH [just did a 'save as epub' on it, nothing else] and I had a hard time getting it to load and eventually loaded it in three smaller files. Does anyone else have any problems with large files? While it is the largest, it certainly isn't the only one that might have issues... There's several others on the 'really really long' list...

Carol

Kinda like Masques? *g* I know there are others longer now, but that one was THE long one for so long it always comes to mind. (You're immortalized in memory, Labby. *g*)

As for getting files to load faster, splitting the internal XHTML sections is the key. Many conversion programs can be told to automatically split it every so many kb, but you want to split at chapter sections, really. Short chapters will load much more quickly. (And they mean short in terms of book length chapters, which is longer even than I've been splitting here.) So as I'm converting the epubs, I split at every chapter so each chapter is a separate XHTML file. The reader won't see it to know it's not all one file--the epub structure is *designed* as a zip containing the XHTML files, images, and even embedded fonts (I don't think we need to mess with those for this purpose, lol)--but it'll load faster than if it were automatically converted to an epub with a single large XHTML file inside.

So anything I create *should* load fairly quickly (though sheer file size will have an effect--a 2 mb epub will take longer to load than a 20 kb one), but let me know if you have problems with it.

(And I've gotten started on your fics now, Carol. Glad you're excited! I'll upload them all when I finish them.)

LabRat wrote:

Sorry for the delay in getting back to you on this -- it's a very interesting idea.

It probably would save a few steps in epub creation if we standardized on a word processor format, but I think the larger picture is how you, the general editors and the writers interact. We definitely don't want to throw a monkey wrench into a system that works. That said, if you all are looking for a change and are interested in moving away from the text-only format, that's something we can explore.

If you want, LabRat, you could start sending me the weekly upload stories in whatever format you receive them in from your general editors. Or if you do some editing on them after, you could send them on to me in Word. I can convert copies to text for posting on the archive and keep the originals on the back burner for conversion experiments.

It'd be nice to standardize on something that preserves basic formatting (bold and italics), that eliminates the need to worry about line breaks, and that poses no compatibility problems.

Actually I think MS Word would be a good choice if we were to standardize on a file format for story submission. Not Word itself, necessarily, because Word is expensive, but the Word .doc format. I think it'd be a bad idea if people felt they had to buy a copy of Word in order to write fanfic. Fortunately lots of word processors these days can read and write Word .doc format -- even the free ones.

I'm curious about the compatibility issues you mentioned, especially involving problems with story files in different flavors of Word. Is that a problem? Files saved in Word 2010 vs. Word 2003 vs. Word 97, etc.?

I definitely do not want to make your job as editor-in-chief more difficult, LabRat. Or that of the general editors.

Would changing things make it easier? And are there program features not being used now but which could improve the experience for writers and editors? Redlining/revision tracking, for example?

Any writers or general editors, if you're reading this, have you experienced any compatibility issues involving your word processor of choice when you've sent or received story files? Is there something we could do or decide upon that would make your job easier? What are your thoughts?

Speaking of free word processors that can read and write Word .doc format (from a few paragraphs back), just to throw out some options...

* OpenOffice can read/write Word docs in a variety of flavors (Word 2003, Word 97/2000). It's not a Word clone, it's a suite -- an MS Office clone. Well, not a clone exactly, but very much a work-alike. Available for Windows, Macintosh and Linux. OpenOffice Writer can save documents as PDFs without requiring Adobe's expensive PDF creator. Its native format is an open document format that allows programmers to easily get at the file's innards.

* AbiWord can also read and write Word files. Available for Windows and Linux. AbiWord is small and fast and speedy even on older PCs. The download is under 10Mb. It feels more responsive to me than OpenOffice Writer.

* Google Docs is for writing in the cloud. If you've got a Gmail account, you've got Google Docs. It can "upload" and "download" files in MS Word and OpenOffice formats. You edit and save your Google doc files online, so they're out of harm's way in case of hard drive crashes. Google Docs also allows collaboration -- up to 50 people can work on a document at the same time! Can you imagine? I wonder what it'd be like to use it to write a story with several other people in real time. Chaos? Genius? Any authors out there willing to put their heads together and give it a try -- and then report back on the experience?

Other options I've read about but haven't tried: ZoHo Writer is part of a large online suite of tools and also allows collaboration. Microsoft's Office Live is supposed to allow online collaboration, but I'm not sure if you need to have purchased a copy of Office to use it. I've heard some buzz about other Microsoft tools in the cloud too.

My personal preference for my own word processing is either AbiWord or OpenOffice Writer. I know there are some quirky things about OpenOffice writer, and that it's a little clunky compared to Word, but it does most everything I need it to do. Except let you find and manipulate paragraph markers, like Word can with its ^p. (AbiWord can't do that either.) But I've found other ways. The text editor Notepad++ -- also free -- is great at searching/replacing returns and tabs. (Instead of Word's ^p and ^t, it's \n and \t)

My favorite text editor is an old program, Arachnophilia. The maker stopped programming for Windows and does a Java version now, but the interface changed so drastically I stayed with my old tried-and-true. You don't hear much about Arachnophilia 4.0 these days, but it's still my go-to tool. It deals effortlessly with paragraph marker and tab search/replace operations. It's not freeware, though, it's careware. You have to be a nice, non-whiny person for a day to use it.

I've always been interested in free and open-source software, but ever since my laptop was stolen a few years ago, I've been almost obsessed with it. Much cheaper to replace software that didn't cost anything in the first place. I lost Quark XPress when my laptop was stolen. It was an older version I registered off a British computer magazine cover disc, so it was sort of free. But it had a one-time registration code and I wasn't able to re-register it. Buying Quark would've set me back $800+. Anybody know of a good free/open-source desktop publishing program?

LabRat wrote:

I'm sorry the macro didn't work for the newer Word. Don't you love it when software makers "improve" their program, breaking something you liked about the old version? That happened to me with Paint Shop Pro. Version 7 -- ugh. Version 6 -- sweet spot.

Karen wrote:

Definitely something to add to the to-do list! Thanks for the suggestion, Karen. For long streches of reading, I prefer reading light text on a dark background and use EditPad Lite for reading stories offline. It lets you configure font styles, sizes, and foreground and background colors. Something like that would be very cool on the archive, and very restful for the eyes.

Speaking of space issues, when I logged into our control panel the other day, and it's been a while since I'd logged in, one important piece of information was missing -- the part where it shows you how much space the archive is allotted and how much is left to use! The contract with our original Web host allowed 250Mb for storage with unlimited bandwidth, and we'd been inching closer to the limit. I called to find out what was going on, and it turns out we've been put on a new plan -- unlimited space, unlimited bandwidth. So, no more nail-chewing over space. We've got room for multiple story formats.

Thanks for the tip on ezPDF too. It sounds like it does exactly what we'd need. It might not be a problem for us if every file is formatted the same way.

Some epub news: Thanks to Doranwen and bobbart, there are 50+ epubs now on the archive. We'll open it up soon. Watch for an announcement probably sometime this weekend.

Best wishes,

Lauren

Wow, great to see this progress

A couple of words on formats, etc.

First off, I’m pleasantly surprised that Word’s classic doc-format is supported by more than Open Office. I figured we would need to use RTF instead to make the Archive-upload open to anybody. Of course, there’s always the fallback-to-txt option in case of a non-standard word processor. As for the different versions of Word, when you select the doc-format, it doesn’t matter whether you’re using Word 2003, 2007, or 2010.

Change tracking is a feature I’ve used with all stories I submitted to the archive so far. Don’t know how much this has actually become a defacto standard between GEs and writers, though. I’m sure Labby could make a poll with his GEs

As for the original upload in a non-ASCII-txt format and the subsequent conversion to ASCII-txt. I’m all for doing it this way, but I believe there is one style-issue we should be aware of. Right now, the Archive presents all stories in plain-text ASCII. Well, duh, because that’s what this thread is all about. But there’s the matter of the typographic characters such as slanted quotes, the ellipses character (which is a stylistically/grammatically suspect entity in and of itself), and the n-dash and m-dash which look indistinguishable in plaintext. In a lot of stories, those characters get replaced by standard-ASCII characters, e.g. using two or three regular hyphens instead of an m-dash. Whether it’s the authors doing this or our dedicated GEs, once we switch to uploading the stories in a richer format, I believe we should standardize those conversions.

Michael

I've some experience with Word compatibilities. Those of us with System 7 Word 2007 (.docx) must save to Word 2003 (a compatibility feature available easily). Unless you have Word 2007, you cannot read .docx. My recommendation would be using Word 2003 as a standard (Windows XP).
When I submitted to the Archive in txt, my GE and I communicated changes in Word 2003. txt is just too hard to read and correct. Then she converted the changes back to txt for the Archive. Eliminating all that would be easier for Word users.
Keep up the good dialog.
Artemis

I've been converting stories (Just a few. Doranwen has done most of the conversion work.) and I have had a change in opinion about splitting long stories.

When I started, I was thinking that the epub conversion editor should "bend over backwards" to put the breaks where the author wanted them. If there weren't chapter breaks in place already, we should go back to the original posts and/or contact the author.

That was how I started. Over the past few days I've realized that would make the process too onerous and will impede the epub conversion process.

My epiphany was that if an author of a long work felt strongly about breakpoints, they would have put in chapter or section markers in the first place. I feel that way, which is why all my longer works are chapterized in the archive.

So, I think that for epub conversion, the rules should be simple.
1. If chapter/part breaks are in place, honor them even if they are small. (Take a look at Meet Me in Kansas City.)
2. If there are no preexisting break points, put them wherever they seem to fit with minimal disruption to the story. If the author (or anyone else for that matter) doesn't like the break points, they can redo the conversion themselves or at least request a reformat. For my part, if I did a conversion and the author wanted it segmented differently, I would be happy to make the change ASAP.

Just some thoughts after a few days of converting stories.

Bob

I do think we should look at the TOCs to see if there is one for the story in question, and if so, follow it--but contacting the author beyond that I'm beginning to think ought to be completely optional.

So maybe we make a general announcement that if an author wants a very old fic (many of which had stuff hosted on the old mbs and the actual story parts are gone so impossible to tell where the breakpoints were) split a certain way, they need to dump it into a doc file and split it and then offer to e-mail it to whoever's converting the fic . . . How does that sound? Put the onus of that on authors, because frankly, with the sheer amount of fics I'm converting, I don't have the time to contact a half dozen authors about two or three stories apiece.

Doranwen, you said: My point is that I don't think that's necessary.

My epiphany, such as it was, was NOT that we can ignore the authors wishes. My belief is that the lack of defined breakpoints in the archive is a reflection that the author didn't feel that particular breakpoints were crucial to the story. If they believed that they were, there would be chapter markers in the archive.

I'm not trying to ignore the author's wishes. However, for me, time is something I always have to factor in to an activity. I found this weekend that it was taking me *longer* to try to find those original posts than all of the rest of the time for the epub conversion put together.

So, if I believe I'm going to have to go on a search for the original posts for every long fic I feel inclined to convert, then I'll be leaving this effort to someone else.

Bob

Sentence breaks, paragraph breaks and scene breaks are all matters of grammar. Chapter breaks are a matter of choice. Terry Pratchett only has chapters in his YA books. The main Discworld novels have no chapters. So if the author hasn't defined them the editor shouldn't insert them.

I completely understand. However, if there are not defined breakpoints, there are two problems:
1. Some e-readers - Stanza for example - will put them in anyway. I can make a better breakpoint decision than Stanza.
2. When a document/story/book is large, the e-reader can get sluggish in handling the stories that are not segmented.

I guess it's obvious I got worn down this week with this effort. I believe in it, but at some point it's not worth the hassle. I'll continue to convert stories for my own use. I WILL add breakpoints in long stories (again for my own convenience) and if no one likes my decisions, I simply won't share my work.

Grumpy Bob
(Believe it or not, that is the name on my cubical marker at work. I get ... testy. And when I was younger, I did sometimes take my ball and go home. I better shut up before I get *everyone* mad at me.)

This one thread seems to have become a very large rock rolling down a steep hill. It sounds like you two are proposing to convert the whole Archive in one fell swoop. How about letting the authors who are willing convert their own stories if the process is simple enough? In that way, you are also guaranteed the author's approval about the new format.
Granted there are old stories with the authors no longer active, but two or three people don't need to do all the work. Shouldn't the word be spread wider?
Also you talk of "uploading" it now. My understanding was there was a filter between author and Archive. Where and what is the link?

Artemis

Lauren's been checking over what we're up to, and had us put together a list of the steps we follow (involving Sigil), so anyone else who wants to can convert their stories to as high-quality epubs. She'll be posting that sometime soon so any author can convert their own if they like and have them look really good. So don't worry, I gather that pretty soon it will be thrown open to everyone. She just wanted to make sure they could turn out really nice and the system works fine first.

I don't want to jump the gun on her, so I won't post any links, but she's got a separate system set up where Bob and I have been uploading the finished epubs and putting together a wiki of the ones we've done so anyone could get them. I'm gathering she'll have that released within the next day, so all you FoLCs can go grab epubs to read. Ultimately she'll merge it with the Archive, I think, like the Filename Z page is.

As for the TOC, Bob, I don't think it's that hard to do a quick search. I have two links open--one for the mbs, and one for the old boards. If I hit any trouble finding it there, I just split myself. But I actually find it easier not having to think about where to split. The author DID, in some ways, tell where things should split when they broke it up in sections to post. The ones who never posted to the mbs at all, that's pretty clear they didn't care about splitting, so we can do it where we think best. That said, it's probably not too much of a big deal to split it where you want if you find checking the mbs is too time-consuming (so don't let that keep you from converting some)--but I wouldn't do that with authors who are very much active and might convert their stories themselves anyway.

The big reasons for splitting are twofold, actually. One, on an e-book reader, you can't just drag the scroll bar halfway down the screen to read the last half. Dozens of page turns are time- and battery-consuming (if my experience with my e-book reader is anything like the norm), and anything beyond a few page turns is a pain to get to. Epubs have a built-in TOC system designed to make jumping around more convenient, but you have to mark something with a heading system to get it to work (Heading 1, Heading 2, Heading 3, etc. like in Word). (Unless you're a guru at the xhtml editing, which I am not--it's a lot of coding to do it without the heading stuff.) The obvious thing to mark are chapters or sections, whereupon people can then go to the TOC and pick Chapter 4, or Part Six, as they like.

The other reason for splitting is the physical splitting of xhtml files. It's not a good idea with an epub of any larger than about 200-some to leave as one big xhtml file. You can mark chapter headings within that, though--you can mark chapter headings without splitting physically, but you wouldn't want to split physically without marking chapter headings because physical splitting means it starts on a new page and random jumps to a new page look tacky. The splitting prevents the file from loading sluggishly, or worse, freezing the reader. I have no doubt that an un-split Masques, for instance (or On the Other Hand, which as an epub came out to 948 kb, I found!), would freeze a good number of e-book readers. It's simply a size beyond what they're designed to load into memory, and that's what the e-book reader has to do with each physical xhtml file. Splitting it means it only loads the one xhtml file the reader's on, then when they hop to the next file, it loads that one instead. It'll look like one big file to the reader, just with page jumps, but the difference in responsiveness is huge.

Okay, let me try once more and see if I can manage not to appear to be trying to create problems.

I do almost all of my reading on my iPhone using Stanza. To facilitate this, over the past few years I have converted almost 200 LnC stories to epub format.

When this discussion got started, I thought it would be a chance to share the work I've done. Unfortunately, when I started considering the idea of posting the epub versions of the stories I've converted, I quickly realized that the formatting I had used was too crude to offer to anyone else. I respect the authors and I feel it is being disrespectful to share a version of their work where the format is less than the best I can make it.

So, with a LOT of help from Doranwen and Lauren I've been converting stories. Using Doranwen's approach produces a *very* good looking work.

As far as segmenting goes, it shouldn't be a problem but it is. Anyone that has loaded a large story into Stanza will understand. I have been segmenting all the stories I've converted for a long time.

Believe me when I say that it is easier not to segment. If I skip the segmenting step I can convert almost any sized story in between 5 and 10 minutes. Adding segments will often double that time. Segmenting takes a lot of work. However, it's necessary to have an end-product that is clean enough to honor the authors whose work we so enjoy.

My point earlier is that the segmenting process is not changing the story. When you print a story on paper (which I have also done) you are artificially introducing segments -- pages -- that have nothing to do with what the author intended. To me, the segmenting that we are talking about is nothing more than making sure that when the e-reader handles the story, it neither chokes nor puts a section-break in a place that disrupts the story.

Back in hiding now...
Bob

Thanks for the explanation, Doranwen & Bob. That makes sense. It just seemed confusing without that inside knowledge. Actually, I don't have time to do any converting of my stories now. Maybe later. So if one comes up alphabetically, you're welcome to do it.
Artemis

Artemis wrote:

It will be, Artemis. I think Doranwen and Bob are doing a great job at the starting line, but the race will be a long one. Should be fun too. Hopefully more people in running shoes will show up.

Since Doranwen and Bob both said they had been creating and stockpiling epubs, it seemed logical to ask if they'd be willing to test out the cobbled-together uploader/wiki system and hopefully target trouble spots before we opened it up. For anyone who's been eager to get at those epubs to read, or who's interested in submitting some, I'm sorry for holding out on you. I wanted to make sure the system was workable first.

I've been cheering on Doranwen and Bob -- and it's amazing how quickly the story count is climbing. Artemis has a good point, though, you guys. Please don't burn out. You've got us off to a great start. There's enough there now to keep people happily buried in their ebooks for weeks!

There's an uploader tool that can be used to send stories up to the folder where the epubs are stored. Bob pointed out a flaw, and I'm unsure of its security and need to find an alternative. I'll publicize the link once a safer tool is in place. Shouldn't take long.

How about this: Anyone else who wants to create and submit epubs in the meantime, just e-mail them to me as an attachment at LCFanfic [at] gmail.com

Yes -- an editor-in-chief and her team of general editors, who do a great job!

The situation with submitting an epub is a little different from that of sending in a new story. The epub has already been through the editing process. It's the same words, just different packaging. So I thought I'd expedite things a bit with a direct upload tool, and now I'll be tweaking that part. And by using a wiki page to hold the links, we can collaborate, and I won't be the bottleneck. Needed to do a bit of testing first.

Doranwen wrote:

Exactly! Merging it with the rest of the archive will take a while. I hope people like the wiki approach for now.

Best wishes,

Lauren

Attention, fans of e-books and Lois & Clark fanfic -- 80+ stories are now available to read in epub format. Get them at the archive on our new epub wiki page .


The whole archive isn't represented there yet, but there's enough to keep you busy for a while. Doranwen has been working on an alphabetical approach by author, and Bob Bartholomew has contributed epubs as well. I'm going to post some stories I've been meaning to re-read. And you can become an epub editor too by converting and submitting some epubs!

The idea is that epub editors -- i.e., you -- can upload epubs and create the links readers will use to download and read them. That'll be the plan once the uploader tool is tweaked. For now, if you've got epubs you'd like to submit, please send them to me as an attachment at this address: LCFanfic [at] gmail.com.

I'm not finished working on my part of the "intro to epubs" wiki page, but you know what? Wikis are works in progress. They're ever-changing. And they're all about collaboration -- so if you spot something wrong or that needs elaboration, you can log in and fix it.

And if you're wondering why the wiki pages don't look like the rest of the archive, it's because I haven't figured out the skinning yet. Works in progress, right?

Please give a hand to Doranwen and Bob Bartholomew!

Happy reading,

Lauren