The last couple of months have been busy but I’ve managed to find bits of time here and there to hack on the new AROS hosted port.
My current bus activity is AROS hacking. I’ve actually been doing at least an hour a day for the last couple of months, so I’m making plenty of progress, but I’m off on a long and exciting tangent so it all seems quite different to what I was doing before.
Long ago I wrote a SDL driver for AROS hosted. Back then I wrote about it being slow because of deficiencies in the driver interface that require flushing the output to the screen for every pixel plotted by software fallbacks.
It would appear I’m back in the AROS game for a little while. I got a nice email asking for some help with fat.
I started writing a long post about what I’m working on right now, but its really quite disjointed because I realised I don’t actually have a point to make.
I have a little treat for the adventurous today. AROSWebKit-preview-r30597-20080302.zip [8.1M] Its mostly unusable, but many many people have requested a demo.
A week later: The major new things compared to my last post are the addition of the page title (and progress bar), the URL entry bar, and scrollbars.
A couple of hours work on yesterday’s effort, and we see this: Had I known just how close I was, I probably wouldn’t have even bothered posting yesterday.
Current progress: This is WebKit on AROS rendering a trivial page containing a H1, a H2, a DIV with CSS styles set to force to 100x100 with a green background, and a IMG of a pirate, though thats not working yet.
AROS work has slowed down over the last week. There’s been a lot of email to reply to (I won’t mention the topic :P), and I’ve doing some web work on the side, but I’ve still had a little time to work on cairo, which I’m now calling finished, at least for the moment.
As you know, I’ve been at linux.conf.au this week. There’s a lot of cool stuff happening in the Linux world, and a few of those things really made me ache to grab the code and get hacking on them.
I’m at linux.conf.au this week and because I’m so well practiced at listening to people talk while doing something unrelated on the laptop (thanks dayjob), I’ve got a hell of a lot of code done, making up for the nothing I did over the weekend.
Things got a little slow in the last week. I spent last week tweaking bits of graphics.library and graphics.hidd to force the alpha channel to be set when calling ReadPixelArray() on a surface that has no alpha (so it can be fed directly to a cairo surface with alpha).
As far as cairo is concerned, its backend buffer get/set methods are only required to store and retrieve pixel data in the format requested by the cairo core.
Cairo is working! So far I have RGB and ARGB surfaces working, and so still have alpha-only surfaces and fonts to do, but that is enough to make the majority of the test suite work.
A year ago today I made my first commit to the AROS Subversion repository. It feels like I’ve been doing this forever, not only a year.
With the help of WebKit developers I finally sorted out the crasher that plagued me over Christmas, and now I see WebKit making network requests, receiving data and calling into the graphics code to get it on screen.
Christmas and New Year festivities are over, and I enjoyed them thoroughly. I spent some awesome time with both sides of my family, played some cricket and soccer, played some Wii, ate way too much several times, and scored a nice pile of DVDs and t-shirts.
Hi. I have lots to tell you, but haven’t had time to write it all down yet. But I wanted to share this, the very first web request ever done by WebKit on AROS:
This is week is insanely busy, as is typical of the week before Christmas, so I’ve had very little time to think about code in the last couple of days.
Now that I’ve (apparently) fixed the loader, my mammoth WebKit test binary loads and runs, and so I’ve begun implementing the stub functions with earnest.
I just now have the extensions to the ELF loader implemented such that my gargantuan WebKit test binary loads. It took me a lot of reading and experimenting to figure out what was going on but I got it.
I wrote a simple launcher for WebKit that creates a WebCore::Page, attaches it to a WebCore::Frame, then tries to load the Google homepage with it.
Just a followup about the whole _GLOBAL_OFFSET_TABLE_ thing. Apparently this symbol is provided by the linker, so it makes sense that it doesn’t work since the AROS linker is actually a custom ld script known as collect-aros which doesn’t handle position independant code at all.
I’m a little stuck. Last night I wrote a trivial startup program to make sure linking and calling into WebKit was working correctly:
Today marks a major milestone for the WebKit port. It compiles! -rw-r--r-- 1 rob rob 24782208 2007-12-08 18:25 libWebKit.a It doesn’t do anything yet, but it compiles.
This week I’ve been working on another WebCore dependency, though a little different to the previous ones. To work well, it seems that WebCore needs a threading system.
Late Friday I reached a minor milestone when I got the platform-independent part of WebCore fully compiling and linking. Next up, the tricky bit: the platform dependent stuff, otherwise known as the actual port.
Michal writes about his continuing pain with the Amiga LONG/ULONG types on 64-bit AROS. Some guidelines for types: If you’re writing new code, just use the normal C types, and if you need types of a specific width, look to C99 uint32_t, etc.
I finally finished my dependency porting stuff, with libxml2 coming to life late last night. I haven’t tested it properly yet, as its test suite requires glob(), which we of course don’t have.
Today I finished porting cURL, a library for getting things from the internets (or actually, anything with a URL). Its probably the dirtiest port I’ve done so far, both because the configure script is a mess (it knows enough to know that I’m cross-compiling, but then doesn’t know enough about cross-compiling to do anything other than get in my way), but also because of the bsdsocket.
Quick one before bed: a port of OpenSSL, which is needed for cURL, which is needed for WebCore. It was actually a pretty easy port to make.
The results are in. The browser will be called “Traveller” (that’s British spelling, with two ells). I had already thought of this as a potential name before asking for ideas, and when a couple of people suggested it too I knew it was good.
Just a few quick updates. First, thanks all for your name suggestions. I hated some of them, I loved some of them, and I’ve finally decided on the name.
Ever wanted to name a web browser? Here’s your chance. I need a name now so that I have a way to refer to the whole project, rather than “WebKit port” (accurate until I start work on the chrome) or “browser bounty” (duh).
I’ve been working on a few small things over the last week or two, trying to tie up some loose ends.
Ahh software licensing, a topic I try really hard to avoid. mausle raised a concern with my previous post about ripping code from the Linux kernel for use with AROS:
I’ve been thinking and poking at AROS a lot this week, so I have heaps to write about but haven’t had time.
Compile stuff is on hold for a little while as I wait for answers to my questions: Can’t find iostream on 4.
Quick one. Short story: I want to port WebKit to AROS. It needs GCC 4 to build. It uses C++. AROS doesn’t have any of that available.
I committed sdl.hidd today. Its not finished by my own standard but I’m pretty much out of time and frustrated by the fact that to make it go any faster I have to keep copying code from the bitmap baseclass.
Not much to report over the last few days, but I have been hard at work. sdl.hidd is functionally complete, the code just needs a little cleaning and commenting before I’m ready to release it.
On the way home today I started poking at the bitmap class, looking for fallback methods that were calling PutImage() or PutPixel() multiple times, triggering multiple flushes and slowing things down.
I have no motivation to blog at the moment, but I’m getting lots of code done. Latest screeny: I’m opening windows and typing things.
In keeping with my recent theme of showing a picture with no actual content, here’s my results from five minutes ago:
With a few minutes to kill after my shower and before sleep, I hardcoded the masks and shift values to get this:
So after a couple of days of wondering why every pixel I got handed by graphics.library was either 0x0 or 0x1 (ie black or so close to black that it might as well be black), I looked at my pixel format setup code and on a whim, removed the stuff where I tried to compensate for SDL and AROS colour component masks being different and used them as-is.
As seen on a morning bus trip: Pixel format conversion isn’t implemented yet, so every pixel is white. The weird thing in the top corner is the mouse pointer, and the other bits are disk icons with their names underneath.
Read this comic. Go on, I’ll wait. There’s something the comic doesn’t tell you. The process that makes photosynthesis happen is exactly the same process was that used to design and implement the graphics hidd interface.
Another quickie: hostlib.hidd in action: What this is is a test program that uses hostlib.hidd to load libSDL.so, get function pointers for SDL_Init(), SDL_SetVideoMode() and SDL_Quit() and then calls them to open a window (that big empty one at bottom-right).
So after the compiler shenanigans of last week I finally managed to write some actual code on Friday. I started with just calls to SDL_Init() and SDL_Quit(), but the compile blew up in my face.
At the start of the week I began writing a SDL HIDD for AROS. Currently it does nothing, just prints a debug message when its init is called to show that its compiled and running.
So I spent my bus rides and my evening yesterday writing a whole new mouse input setup for FFE. It works properly, in that pressing the right mouse button stops the system receiving mouse events.
So my interest for this week (I’m fickle so you can guarantee that next week I’ll be doing something else) is porting games to AROS.
I’ve been back working on fat.handler this weekend. I had to look at the code for something and actually found it kind of interesting, which I thought was long past.
Status update. I have a mostly-working test.library that implements a TAP producer. So far it does TAP output, has basic success/fail counters and a couple of other bits.
Its nearly 2am, and I’m sleepy. Just a quick one to give you an idea of where this is going:
In typical fashion, I got bored with the pipe handler. It’ll still have to be done, of course, but a couple of things about it became non-trivial, so I had to start thinking and designing, and thats not interesting or fast, so I couldn’t be bothered anymore.
Today I checked in my pipe code. It consists of the Pipe() system call, the FSA_PIPE and ACTION_PIPE definitions, the implementation in pipefs.
A pipe created with Pipe() has two handles. Therefore, the initial use count should be two, not one. Obvious stuff, really.
I’m having a wonderful time delving into lots of bits of AROS that I haven’t seen before. What started as simply wanting to make IN: work has lead me into the depths of the shell and beyond.
Named pipes are like normal files in that they can be created and deleted. The difference is that when you read from it, nothing happens until something else writes to it.
I started writing an entry about Open() and its edge cases this morning, but it made very little sense, which I suppose is fitting.
So Pavel went on holiday this weekend but didn’t want to hold me out, so he graciously offered for us to switch places - I check in my GetDeviceProc() and ErrorReport() patches, and he’ll take them on holiday with them and update his code to match my changes.
Today I rewrote ErrorReport() . The previous implementation didn’t handle most of the errors it was supposed to (which aren’t actually many), and was in need of the same kind of general cleanups as everything else has needed.
As mentioned previously, AROS DOS has some special magical device names that don’t correspond to any underlying device - IN:, OUT:, etc.
Just found another problem with IOFS. There’s no really good way to determine if two files are on the same filesystem, which you need to know to safely rename files and create hard links.
Quick notes, as I’m nearly home. Open() handles a few “magical” filenames: NIL:, PROGDIR:, CONSOLE: and *. The last two are identical, and should return a handle on the console that the program is running from, regardless of whether or not standard input or output have been redirected.
In our last episode I was implementing FileLocks into dos.library. Well its done now. It took a couple of rewrites (if you can call mass search-and-replace operations a “rewrite”) as Pavel kept pointing out problems with my implementation (and rightly so), but its done.
As noted, I’ve started hacking on DOS. The first thing on my list is to make it use struct FileLock correctly.
I implemented SET_FILE_SIZE late last week. I don’t want to talk about it, read the code if you need the details.
After implementing the global lock stuff last week, I spent a couple of hours on the weekend making renames work. Its a naive implementation, which as we know contributes to fragmentation, but it works and was trivial to implement, which is all I care about now.
Last night I finished refactoring the lock code and checked it in. I’m kinda surprised that its still working. Here’s the story.
Since fat.handler reached a pretty significant milestone I felt like I needed a break before getting into the excitement that is refactoring all the lock code (sigh), so I’ve just finished picking off an item on my todo list - faster bitmap scaling.
Support for creating and writing files was pretty much finished on the weekend, but my power test of copying the entire system to a FAT partition would cause things to explode in myriad ways, but not until a few hundred files has been copied.
I’m currently checking in some fixes that I believe will stabilise the file writing. There was a few places where I’d got the math wrong when generating directory entries which would sometimes result in weird corrupted filenames (depending on what was on your disk before I incorrectly wrote directories onto it).
As of yesterday you can now create files. Its still not perfect - you can break it by copying large numbers of files in one go (eg Copy ALL C: FAT:), which will give you some broken filenames.
As I’ve been refactoring fat.handler I’ve noticed that its gradually change from its original spaghetti into a fairly layered setup - the I/O cache, the FAT, the directory code, the file read/write primitives, then high-level operations code (“create a directory”, “delete a file”, etc), with the OS interfacing (packet handling, volume management, etc) at the top.
I implemented DELETE_OBJECT today. As it suggested, it deletes things - its the power behind the C:Delete command. I’m actually quite pleased with how straightforward it was to put together - it suggests that I have the internal API for updating directories and writing things right.
I put in a marathon day of code yesterday - perhaps six hours by the end - and finally got directory creations working.
I’m about to go out, but here’s a couple of screenshots to demonstrate the progress of the last couple of days.
I’m on my break! Today is just the second day, and still the weekend, so I haven’t really done much of anything yet, but it’ll happen - all this time is making me giggle :)
This morning I checked in rewritten fat.handler directory code that uses the new cache. The code is now much cleaner, and I hope more readable.
Work is absolutely insane at the moment so I come home very tired, which means I’m only getting an hour or two a day to work on AROS.
I finished implementing the read side of the block cache yesterday, but I haven’t even tried to compile it yet. I really don’t like it.
I’m currently on a tangent within fat.handler. I started to rip out the caching, but because I hoped to add it back in later in some form I started making everything that needed disk data call a single function that gets the block from disk.
Life is completely mad at the moment. I’ve still had a little bit time to write code, but work is full on and after hours is crazieness too, so I haven’t had a lot of time blog.
A couple of screenies before I dash home. I added seperate DOS types for the three FAT types this morning, and updated C:Info and DiskInfo to cope, so we get this:
Got back to work today after a few days off. No bus trip means limited dedicated coding time so I’ve mostly been writing email for the last week.
From: Robert Norris To: TeamAROS Subject: Changing the DOS Packets bounty Hi Team, I've reached a block on the DOS Packets bounty that I can't work my way around, so I need this group to help me figure out what to do.
The response to the packet stuff has been great, mostly because people can now read their FAT disks. Its exciting!
Hmm, haven’t posted for a little while. Last night I dropped all of the packet code in AROS SVN. That includes packet.
I mentioned in a comment that FUSE-based filesystems could be fairly easy to port since they’re not in the kernel and therefore shouldn’t have tight dependencies on the kernel VM subsystem.
It seems every week there’s a new discussion about how to bring a proper browser to AROS. I’ve seen talk about how to structure the bounty (if we have one), which codebase to use, whether it the source should be closed or open, and so on.
Now that the filesystem actually works, things have got a little boring. There’s still plenty of work to be done, but not much I can test without getting updated FATFileSystem code.
Just got back from bowling and Daytona. Had a couple of drinks in quick succession so I’m a little buzzed. I’ll write this and then go to bed, its 1am.
We’re in the middle of a heat wave here in Melbourne, as is normal for this time of year. Both Saturday and Sunday were up around 38-40 degrees (celsius), so I couldn’t find much motiviation for anything other than dozing on the couch and complaining.
[packet] in init [packet] in open [packet] devicename 'fdsk.device' unit 0 dosname 'PKT' [packet] couldn't load fat.phandler [packet] couldn't load L:fat.
No huge progress, but a few small things to report. I’ve got FATFileSystem building the way I want it. Turns out the whole main() thing was totally unnecessary - its enough to build with usestartup=no uselibs="rom".
I consider myself a fairly good programmer, but I always make the most ridiculous mistakes, that usually cost me hours or days in debugging.
I spent a couple of days beating my head against the AROS dark magic that holds everything together. I got FATFileSystem building, but on trying to call into it with my loader, I’d get a segfault every time I tried to make any kind of function call.
packet.handler is coming along, but of course I hit yet another obstacle today. The loading mechanism I described previously is fine, except for a fatal flaw: OpenLibrary() loads shared libraries, meaning that I’ll only get one instance of the packet handler ever.
I’m making good progress, and am quietly confident about success. I can see the next few steps I need to take, which always motivates me, and usually means I’m on the right track.
I don’t feel like I’ve got much to write, since I’ve spent the weekend just reading code and getting more and more confused, but Tom_Kun (of AROSAmp fame) told me to just write about the confusion and bemoan the lack of documentation.
As you probably expected, I’ve finally applied for the DOS packets bounty. While is hasn’t technically been accepted yet (since the Team AROS mailing list is having some issues), I’m working on the assumption that it will be accepted, and starting work accordingly.
Sheesh, you step out for a couple of days and people start hassling you to write (thanks Christoph ;) Anyway, I’ve sort of backburnered filesystems for a little while.
Michal Schulz informed me of the existence of fdsk.device, which is basically a loopback device - a way to mount filesystem images.
Michal just sent me a read-only FAT32 driver that was written by a Marek Szyprowski for MorphOS. Its DOS packet based.
My brain isn’t in the right place for SDL hacking right now. It seems pretty doable, but I’m bored. I keep thinking about filesystems, so thats where I’m going to focus my efforts for the moment.
tap.device is pretty much finished. There’s a few little things that need to be added, but my motivation is gone on it now - I’ll just add bits as they’re requested.
Finished off the stats tracking code this morning, which gets the thing into a usable state, so its time for a release!
Does this mean anything to you? rob@plastic:~$ ping -c5 192.168.30.2 PING 192.168.30.2 (192.168.30.2) 56(84) bytes of data. 64 bytes from 192.
Someone in #aros was saying that AROS has no goals and no directions, and that he couldn’t support it based on that.
I’m unbearably close to having this working. I found the problem that I described yesterday. I was using soft interrupts to have UnixIO signal event readiness.
Work continues apace. Yesterday tap.device received and decoded its first packets, and I happily watched as AROSTCP issued write commands in response.
Made some excellent progress yesterday. Turns out that only code built into the kernel can access the host OS, so I have to make use of a HIDD of some kind.
So I got a stub driver done, and this morning instructed AROSTCP to use it, but it failed in startup - couldn’t find tap.
Merry year, etc. I didn’t write a great deal of code over the break, opting to trounce Dark Samus instead. But now I’m back at work, which means a few spare laptop hours each day, mostly while on the bus.