Lois & Clark Forums Forums Fan Fiction Fanfic Related Epub conversion project - chapter splitting & fics in process

I don't see any other thread that's doing what I think one needs to be doing, lol, so I'm creating this one.

This can be a general place to announce stories you're converting: "I'm working on this really long story right now, so you can know not to start on it so you don't waste time" or it can be a place for authors to volunteer that they'd be willing to tell converters where to put breakpoints for chapters, if the TOC isn't available (which is the case with many of the older stories). (If there should be a separate thread for the author bits, go ahead and create one; I wasn't sure and I'm not an author. )

I'm going to use the first post to say that I've been itching to work on Nan's stuff for a while (I have fond memories of reading the Dagger series for the very first time when I had just gotten into L&C fic), so I'm jumping ahead and starting on them now--slowly. *g* My job is taking a bit more time than it used to since I got promoted, so I'll be slowing down. I'm going through alphabetically, so starting with Assassin's Dagger for now.

If you'd like to work on her stuff too, I'd welcome any help on the Home series you'd like to provide.

Anyone else working on converting fics to epub format right now? How are you finding the process? Any epics you decided to tackle? (I'm hoping I don't run into any super long ones for a bit--Learning to Love and On the Other Hand were monsters!)

I'm working on my "All the Daytime and the Nighttime." I got the extra line stops removed, but the lyrics to the songs are smooshed, so I'm now working on fixing that. I don't have any epub conversion software, so I was hoping to send that on to you or Bob. I don't think I'm clear on the idea of "chapter splitting". My stories are in chapters. In the commercial epub books I'm reading, the chapters are delineated and the text flows fine, so I don't see where that needs extra work.

Also I can give PDFs of all my stories anytime you want.
regards
Artemis

Well, lets see.

I started on Nan’s Home series but stopped after “Home” to see about split-points. I have moved on to other stories so for now all of the Home stories other than the first are “up for grabs.”

I went ahead and converted the Nightfall series in case anyone wanted to read it while Nan is working on Smallville. However, the break points in those stories are MY best guess and they may get re-done. Other than the break-points not necessarily being in the right places, they are in good shape.

I am working on two of Poussin’s stories, “Somewhere…” and “Under the Light of a Dead Star”. Those should be done by tonight.

Other than that I don’t know what I’ll do next. I will take responsibility for converting all of my own stories. . My plan is generally simple, other than my own, I’m converting the stories that were in my eReader when this started. (So don’t be surprised if there is something of a WAFFy tilt to the epub library for a while.)

If more people get involved, we can worry about a more formal plan. For now, I’m sort of free-forming this. However, I’d love to see enough people get involved that we did need to use this thread to plan. I’m happy to move to a more structured approach if necessary.

Also, if anyone has particular requests and feels that they may not be up to the task of a conversion, this might be a good place to post requests for conversions of particular stories.

Bob

There's two parts to chapter splitting. One is marking individual chapters so the TOC will pick them up and readers can navigate to those chapters specifically (a crucial aspect when reading on an e-book reader where it's difficult to page turn 10 or 20 times in a row--mine has physical buttons or a jog wheel, not a touch screen, so I'm trying to aim things so it'll work well on ones like mine as well as the touch screen kind, where scrolling is a lot easier). The other part is splitting the physical xhtml files so they won't be too large. For super shorts this is clearly not necessary. For some larger shorts this is a good idea just to prevent slow loading times. For epics, on the other hand . . . it's a necessity, else some readers may crash altogether! Shorter files will not hurt, so erring on the short side of sections is a good idea. I've been splitting files upwards of 20 kb, mostly, though only for sure anything larger than 40 kb. Depends on the fic. Smaller than 20 kb is definitely short enough it doesn't need any splitting.

Both of these aspects of splitting go best hand-in-hand--though it's easy enough to mark a chapter in the TOC without actually splitting a file physically. (It doesn't look so good to do things the other way around, because physically splitting a file creates a page break in the book. That's what the reader will see--best to have those page breaks at chapter breaks as well.)

It's quite easy to do this in Sigil, actually--just a few clicks and mouse dragging to select at one point. Why don't you download it and give it a try? I was a little confused on how to use it at first--until I took one or two glances at the user manual and discovered it was actually super simple. They designed the program for us ordinary readers to create e-books from, so we didn't have to be html gurus to turn out great epubs. I think our instructions on the wiki are fairly clear, but if you get confused on 'em, let us know and we'll tweak till it makes perfect sense.

And while I wait to see if Nan wants me to split with her breakpoints, I'll get a few of her really short fics done. Before the Beginning is now up. (I'd forgotten about that one! Was nice to rediscover.) I have done way too much re-reading while converting, I must admit . . .

I just added Pam's Tryst and Merry Trystmas.

Things have slowed down a little bit but we are still moving forward. I just counted 112 stories.

Again, if anyone has requests please post them.

Not sure what I'll be working on next. I'm still reformatting my own library of stories.

Bob

I'm still working on Nan's. New job promotion has kept me busy this last week as I now work fulltime. I'm trying to get at least one story a day more or less. Working my way through the D's of her fics right now--just uploaded the first two Dagger stories as well as the little vignette right after them.

I just added Shayne Terry's " Family Hour ." Great story! Probably the fourth time I've read it.

Used Sigil to create the epub, and it was my first time using it to split a large story and create a table of contents. Finding the story's TOC thread on the forums here was a great help for locating the split points. Thank you, TOC forum editors!

Like Doranwen said, reading the Sigil online docs is helpful. Piece o' cake.

And the Firefox epub plugin Doranwen mentioned works great too.

Lauren

I just added Carol Malo's " Strange Visitors ." One of my favorites.

I tried something a little daring with this one. Carol noted that "words between * * denote emphasis." Because epubs allow formatting using italics -- and of course bold and heading levels, etc. -- I used italics. Something you can't do in plain text, as all authors know.

(Carol, if you read this and want your asterisks back, I'll put them back in for you.)

Also, I did "soft splits" for the TOC section breaks by adding blank Heading 3s: Instead of <h3>Part 2</h3> I used <h3 title="Part 2"></h3> in the code view, and Sigil picked up the attribute of "Part 2" for the TOC. But you won't see "Part 2" while you're reading the story.

Fun, fun!

Lauren

Ah, so the reader will just see the story jump pages when reading (but have the TOC behave normally) or did you not section break it? For small stories it's probably not necessary, as most e-book readers *should* have enough memory for that! I like doing the splits anyway, as something about it feels like a "real book" to me--and it's so fun to read chapter by chapter. I've done some code editing too, but it's so much easier when I don't have to enter alternate titles, lol, even if I know how.

I've done some italicizing in discussion with Carol M--she actually said it would be best if I could for certain of her stories (some flashbacks and stuff), so I did. I could even put in the pictures from Photo Shoot like it was originally designed to be read . . . I wasn't sure about that so I didn't try (inserting pics is not something I've tried yet), but if that is something that would be good, let me know.

Only minor update tonight--am down to Fly Hard Revisited now. It'll take a bit to finish Nan's at this rate, but I'm going to plow through it in spare time, head to Caroline K's for a quick break . . . and probably by that time the nostalgia factor will kick in and I'll have to get to Wendy's. *g* *fondly remembers the days when she had JUST discovered L&C fic on Fanfiction.net and was reading through Nan's and Wendy's fics as fast as she could*

Doranwen wrote:

I've been putting section breaks and TOC splits at the same places, for the most part. Just thought it was fun to make TOC labels that show only in the TOC and not in the story. Though if authors label a section (Part 1, Epilogue, etc.), I make sure their label is visible in the story.

Re: italics, lots of authors have been using asterisks over the years in lieu of italics. And all kinds of symbols to represent things like thoughts (though in printed material that's usually italics too). It makes the story look a little more interesting, formatting-wise, to use italics. Carol's story was short, so making the switch was easy. With longer stories, I gave up!

I'm kind of leery about using images, especially stills from the show, on the archive. It's something that's gotten some other fandoms in trouble with the powers that be, and I'm paranoid. Would be cool, though. If some other site wants to host them, and the author's agreeable, it'd be something to pursue.

Nan and Wendy are great ambassadors for our fic-fandom. The story that got me hooked on Lois & Clark fanfic was the first I'd read -- "Counter Clark-wise" by Zoomway. Fantastic story. Was truly addicted after that.

Speaking of "Counter Clark-wise," I've added it as an epub, along with:

"12:01" by Zoomway
"Aliens and Strangers" by Susan Stone
"Something the Cat Dragged In" by CC Aiken
"Cheese of the Month" by Susan Young
"My Adventures With Superman" by Kathryn Ann Kent
"A Shot in the Dark by Sheila Harper"

Best wishes,

Lauren

/nods knowingly/
And it certainly doesn't help that formatting something using italics in Word and for posting on the boards are two very different things, thus making it quite impractical to create the source with proper italicization.

On a similar note: for practical purposes (basic ASCII characters only plus because it looks better in plain text) a lot of stories get submitted to the archives with their nice, typographic quotes replaced by the generic ASCII ones. Another potential nightmare to get those back

Michael

You might try http://wmpoweruser.com/freda-new-epub-reader-for-windows-mobile/

Ahh, I see. The only reason I haven't considered doing that is because when you do a section break, it starts the new section on a new "page" for the reader. So it would be like having random blank space and jumping to a new page, but without a title to go with it. Since that looks kinda odd and I want things to look good as well (plus doing the editing in the code to get the TOC labels to be different from what's in the story is a hassle for me, lol), I decided it would be best to just create labels that didn't exist in the original story. (Though most of 'em are straight out of MBs, which really helps when it comes to breaking things up.)

Just finished Home: Circle of Fate--I'll have it uploaded and edited in a jiffy.

Thanks for thinking of my story, Lauren. I'm not committed to those asterisks so sounds fine

carol

For my part I’m not changing things like asterisks. My reformatting has been simple and minimalist.

I have added the words “Part x” at the split points. I like this because I can scroll up to the top of a section and see where I am. I’ve looked at some of the ones without this and find that I get “lost” as to my location. Not all readers make the currently active section obvious. Anyway, adding “Part x” is all I’m doing in terms of modifying the document.

I have gotten frustrated by the TOC problems. Stanza on my iPhone seems particularly flaky about TOC navigation. Sometimes it seems like it doesn’t like the approach I’m using in large files but later I’ll go back to the SAME file and it will navigate fine.

I’ve considered making all of the headings (title, author’s name and section breaks) level 3 to see if this make the problem less obvious, but I haven’t tried it yet. The format I’m using (the one described in the help screen on the wiki) produces a format that I like and seems to work perfectly on all of the PC-based epub programs.

Now, if someone knows of a formatting approach that works better on real e-book readers, then I’ll be happy to change. For me, portable readers are the “real” target environment and we should try to make things look as good as possible for those devices.

Bob

I can say that other than leaving the title in the TOC of the multi-part fics (which just adds a unnecessary second layer of TOC on my device), the process we've been using looks great on my e-book reader--the Astak EZ Reader Pocket Pro. (The different heading sizes make it look really neat!) But mine isn't the only kind out there. Anyone out there with a Nook or iPad? Anyone tried converting the epubs to mobi and trying them on the Kindle? We'd love to hear how your experience with these fics is like.

Just read Accidental Husband on my iPod Touch and it was fine.

Thanks! Will have a look

Michael

Michael wrote:

I see what you mean. For the boards though there's a roundabout way you could use. If you saved your story as HTML, you could use a text editor to replace the HTML <i></i> tags with board [i][/i] tags -- and get rid of practically all the other tags. I wouldn't recommend trying that straight from Word, because Word makes such ridiculously marked-up HTML files that it's impractical to try to edit the code manually. AbiWord , however, saves very clean and simple HTML if you choose "Export as HTML 4.01" as your only export option.

Have I mentioned how great AbiWord is? Just noticed that a recent version has added OpenOffice to its import/export list of file formats.

Now I'm feeling guilty for destroying your smart quotes. I think we started doing that because they displayed as funky symbols in ... heh, can't remember now where the funkiness was spotted but definitely remember seeing it. Have to admit I haven't seen many funky characters lately. I bet a good search/replace routine could get the curly quotes back -- (space)" = opening, "(space) or "(punctuation) = closing, etc. Or it could be a nightmare.

Thanks! I hope the reader dcarson recommended works out for you.

Doranwen wrote:

The blank pages thing happens to me a lot with Stanza on the iPod Touch reading books from Project Gutenberg, and I got used to it. But I guess it is annoying, so I'll stop. Certainly easier NOT to format that way. With older stories there's no TOC to go by, so I've been looking for authors' section breaks (usually marked by *** or something like that), especially ones that are kind of provocative or "cliffhanger-y," to make the breaks in Sigil.

Speaking of Sigil, Doranwen, I just noticed that when I read epubs I've formatted in desktop Stanza, I don't see chapter headings. But when I read epubs you've formatted, I do! Still seeing them in iPod Stanza. Can't figure that out -- will need to reread your steps in the wiki to see where I'm going astray in Sigil.

Carol wrote:

You're welcome, Carol. It's one of my go-to fics. When I read it again this line of Jimmy's had me chuckling: "You wouldn't believe how stubborn she is. I don't know what she wants, but you're probably gonna do it for her." Love it!

Thanks for cutting me slack on the asterisks.

Bob wrote:

Excellent point.

And with me it's the desktop Stanza. Weird!

Best wishes,

Lauren

That's just the thing, Lauren, in plain-text view, the plain quotes look better. The slanted ones are too thin when viewing the file in a browser. IMHO

I believe nightmare because of the ridiculous amount of edge cases. It starts with the beginning of the line, continues with the subject of m-dashes, then there's quoting inside a quote block. And what do you do with the apostrophe vs. regular single quotes. I *think*, writing a parser that counts quotes is the safer choice. At least then you don't have to put everything into one search&replace. I once reverse engineered punctuation using a set of regexes. Got about 95% of the stuff right. Because 80% followed a simple scheme and I used a diff-tool to find obvious flaws. But maybe I'm too much of a worry-wart and natural language in fiction is much uniformer than that But hey, I could always try to copy/paste to AbiWord. Maybe the formatting survives the clipboard. But to be honest, for the extraction of simple text suited for novels, an XSLT might be the much funner choice. Then I could just build a small application where I just drag&drop a docx and get the UBB formatted result... And it's reusable.


Michael, who now has to deal with a rampant nerd-muse in addition to his comatose fic-muse. And Nerd, well, he thinks Muse looks cute with her little wings and all and now I have to keep him from waking her and distracting her into the nfic side of things.
*arg* Muses...

For what it's worth (or shall I say I'll put my two cents in? whatever expression you'd like ), I prefer the straight quotes over the curly ones, because they copy/paste to more places with less hassle. And I think it would be a nightmare to convert *back* to curly quotes, as Darth Michael showed by the work someone would have to do to make it work right. (Simple find-and-replace wouldn't work, that's for sure!) But I'm just looking at it from a "here's where it is now" POV--if the archive switches the default file type, that's another matter altogether.

Just finished the Home series finally! I forgot how prolific Nan was until I tried this. It's taking an awfully long time to get through all her fics. *g*

Sorry to be late replying, but Michael's last post sent me scurrying around to wikipedia, w3schools, php.net, and down the rabbit hole. Is it still 2010?

Michael wrote:

I agree they look better in plain text, though with a proportional font they'd look nice.

Doranwen wrote:

OK, straight quotes wins.

Michael wrote:

I think I understood about half of that. Regexes? Reverse-engineered punctuation? Great shades of Elvis!

Hey, you use what ya got! AbiWord is my hammer. One of 'em, anyway.

I was thinking more like save to doc from Word, open in AbiWord, save to simplest HTML, open HTML in text editor and search/replace i's in angle brackets for i's in braces...

... well sure, if you want to get all high-tech and super-efficient about it. :p Seriously, Michael, I'm in awe of your mad skills.

Lauren

... well sure, if you want to get all high-tech and super-efficient about it. :p Seriously, Michael, I'm in awe of your mad skills.

Aaaanyhow. If there's interest in a nice, little DOCX-2-UBB converter for simple documents (just text, italics and bold) I'm quite willing to share Just have to finish it first. Oh, and of course, it's also suited for DOCX-2-Archive.



Michael

Just a little update to let everyone know I *finally* finished converting all of Nan's stories (at least, the ones that are on the Archive) to epub! I got a little sidetracked, what with life being busy (and discovering my new favorite TV show, Sue Thomas F.B.Eye) . . . but I haven't forgotten this project! I'll keep working on fic conversion as I get the time.

(I need to keep a better eye on the What's New page--I totally missed Games People Play until I double-checked at the end to see if I had gotten all her fics converted.)

Taking a break from nostalgia and going back to the alphabetical order, I'm going to tackle Caroline K's fics next.

I finally pulled myself away from reading Sue Thomas fics and worked on Caroline K's fics tonight. Not too many, so done in one fell swoop. Enjoy!

While I converted, I read them, and had to chuckle at several funny lines:
"Do you have a headache?"
He let out a breath. "I certainly do. I found it on my front steps."
and
"It wasn't quite a date, but it was a lot more fun than tonight's dinner with Mr. I'm-So-Rich-and-Charming-and-Important."
"Did you just say *fun*?" Lucy stared at her. "I'm sorry - I was just talking to my sister, Lois. Did you happen to see where she went?"
*giggles* (And on another thought, how come all the Lucys in shows I've seen turn out to be matchmakers? Something about that name . . .)

Anyway, I shudder to think how long it will take me, but I'm going to convert Wendy's next, for nostalgia's sake (hers and Nan's were the first fics I ever encountered in L&Cdom). Considering how many there are, and how LONG every last one is, lol, I suspect it will be awhile.

(On an interesting note, my e-book reader recognizes bold and italics just fine in epubs, which is what we're creating, but NOT in html files. How curious . . .)

Doranwen, congrats on finishing Nan's stories! Look's like we're up to 220+ epub files now. Wendy's stories will probably take a while but it'll be nice to have them as epubs too.

It's been a while since I've checked in. Here's what I've been up to....

I've been reading about XML and XSLT (thanks, Michael! ) along with PHP. Still a slog, and still working on it, but I'm also tackling the problem from another angle. I've been going back to the oldest stories and putting them in our current format manually. Still got a ways to go with that, but I figure having all the stories consistent will simplify things later. Because those old stories certainly weren't consistently formatted -- it's worse than I thought! I bet you're thinking, "Well ... duh!"

Also, Doranwen, you asked if story filenames are duplicated, and I said no, but unfortunately I've found at least one case. What I didn't take into account was that the filename catalog is incomplete -- stories are missing from it. And I'd been checking new filenames against the filename catalog. Working on that too.

Still loving iPhone/iPod Touch Stanza for reading. And there's a newish feature in iTunes -- or it's been there all along and I missed it (argh!) -- which simplifies the process of getting fanfic epubs into Stanza: File Sharing. I added a how-to page in our wiki to show how it works.

Current problem is ... what to do with all that leftover Halloween candy. It's a good problem to have.

Lauren