Towards a Reopened CFNetwork

I think one of the coolest things about Apple, Inc. in their modern incarnation is how great a member of the Open Source community they have become. The fact that the true core of Mac OS, the Darwin BSD implementation, is open and available for all interested parties to review, modify, and learn from is unprecedented in any commercial company I am aware of (at least, none that started off life as a closed-source entity.)

In addition to the core OS components from the Darwin layer, they have contributed improvements to the GNU compiler collection, their amazing WebKit rendering engine, spearheaded the OpenCL initiative, shepherded LLVM into an absolutely amazing compiler system, provided a fantastic local host resolution and routing system (Bonjour), and have contributed the equivalent of millions of dollars in development value to the world at large.

Amazing!

So of course I am writing this post to ask for EVEN MORE.

While working on a native Windows port of WebKit (one unencumbered by proprietary Apple libraries that prohibit redistribution), I started working with CoreFoundation. Along with the many other products I listed above, they opened a large subset of the full CoreFoundation product as CF-Lite, large enough in fact to support running WebKit. They even provided useful examples and arguments for basing a porting strategy around CF-Lite. While they make dumps of the CF-Lite source tree available with each Mac OS release, they did not generally release Linux or Windows sources (which had to be recreated externally). They were also uninterested in opening it up as a WebKit-style open source project, so I started hosting them as the opencflite project on SourceForge. We now fully support Mac OS X (a no-brainer), Windows, and Linux.

CoreFoundation is really neat because of its various data types and operating system abstractions, but it also has a nice modular system for modelling various operating system elements (such as sockets, files, timers) as part of a common runloop, allowing easy asynchronous operations. What is even neater is when you can make use of CoreFoundation's sister framework, CFNetwork, which extends this idea to various network features, such as Host lookup, DNS, and Bonjour.

Up through Mac OS X 10.4 (Tiger), CFNetwork was Open Source software. Alas, those salad days were not to last. With the release of Mac OS X 10.5 (Leopard), CFNetwork closed up and has not been seen in the wild since.

Initially I thought this was a mistake, but after corresponding with Apple's Open Source Product Manager I was informed that it was indeed closed on purpose. After a couple of years of e-mailing various parties associated with the CoreFoundation/CFNetwork product line, I eventually filed rdar://7195752 to request the source to be reopened.

I'm not sure why CFNetwork was closed. But even if it contains Secret Alien Technology, I believe that portions of it could be made available in an Open Source form. In my bug report, I gave an impassioned plea that basically boiled down to the following bullet points:
  • Increased uptake of CoreFoundation as a general purpose, cross-platform library would facilitate porting of applications to the Mac OS.
  • Security engineers would be able to audit and validate the source code, increasing overall platform security.
  • Presence of the source and examples on more development platforms would increase developer mind share, and help extend the visibility and utility of Apple development tools.
  • More eyes looking at the code would benefit the source base as a whole.


As I conclude in my bug report, I sincerely hope that Apple could be convinced to open source at least portions of CFNetwork so that its use could be extended outside of Apple. This would greatly aid in my efforts to keep the Windows Cairo port of WebKit in sync with the full Apple version, as well as allowing work on the PureDarwin project to continue.

Comments

Eike Hein said…
It's worth noting that "their amazing WebKit rendering engine" started out as KHTML and KJS from the KDE project.

As for closed-source entities opening large bodies of code, how about Sun? Solaris, StarOffice (OpenOffice now), Java.
davec said…
Any luck or progress on CFNetwork for Windows? Couldn't we just take the old 10.4 CFNetwork and fix the bugs and use that?
davec said…
still wondering if you consider the 10.4 source code good enough to start with?
davec said…
If you still care about this project, i have gone ahead and hired someone to do it. I will then release it to the OpenCFLite project. Stay tuned.
bfulgham said…
Awesome! Sorry I didn't notice your earlier note. I think the 10.4 sources are a decent place to start, but it might be more efficient to build something like a wrapper around libcURL.

The existing cURL implementation suffers from poor access to cookies, so writing this library just to get better access to those routines would be a big win.

How are you going to handle SSL, etc?
davec said…
currently i'm bailing on security. i only care at the moment for the ability to create a simple web server on a high numbered port, and the ability to download files.
i had thought that if it were necessary we would use boost::asio, but only if absolutely necessary.
davec said…
The first part of CFNetwork for Windows is done. this is the "server side" functions, to help you create a server (including bonjour publishing). if you like, i can send you the code. however, the "client side" functions do not work yet. this second half of the project will be started the first week of April and will likely take a month. it will include downloading, SSL, redirects, bonjour browsing, and possibly ftp downloading. would you be willing to provide a code test case for getting access to cookies thru CFNetwork, as i am not familiar with that
Anonymous said…
You may be happy to know i've actually now got CFNetwork mostly working on windows, including some SSL. There's more to do but i think it will be done in a week or two

Popular Posts