Planet Voidspace

October 07, 2008

Techcrunch

Netvibes Partners With Russian Web Portal Rambler.ru

Netvibes, the site that lets users customize their homepages with a variety of widgets, has partnered with Rambler.ru to bring its widgets to the massive Russian web portal. Rambler is the Yahoo of Russia, with an estimated 40 million users and 3 billion monthly pageviews. The deal is being described as “multi-year” and worth “multi-millions”, but further details haven’t been disclosed. Netvibes availability on Rambler.ru is expected to begin in November.

This marks the first time Netvibes has licensed its platform for installation and distribution to an independent third party, and probably won’t be the last. In order to stay competitive with other widget hubs like iGoogle, Netvibes would do well to spur its growth by offering its widgets to other region-specific portals (that said, Netvibes has been doing well, with a reported 500 billion widgets served montly). According to the press release, the Rambler homepage will include Google Search, Blinx video search, and a number of Russian services like Price.ru.

In July Google acquired Begun, a contextual ad service, from parent company Rambler. As part of the $140 million deal, Rambler has been using Google for some of its advertising and search functions.

Crunch Network: CrunchBoard because it’s time for you to find a new Job2.0

by Jason Kincaid at October 07, 2008 03:12 AM

Ian Bicking

The Philosophy of Deliverance

I’ll be attending PloneConf this year again, giving a talk about Deliverance. I’ve been working on Deliverance lately for work, but the hard part about it is that it’s not obviously useful. To help explain it I wrote the philosophy of Deliverance, which I will copy here, to give you an idea of what I’ve been doing:

Why is Deliverance? Why was it made, what purpose does it serve, why should you use it, how can it change the way you do web development?

On the Subject of Platforms

Right now we live in an age of platforms. Developers (or management or coincidence) decides on a platform, and that serves as the basis for all future development. Usually there’s some old things from a previous platform (or a primordial pre-platform age: I’m looking at you formmail.pl!) The goal is always to eliminate all of these old pieces, rewriting them for the new platform. That goal is seldom attained in a timely manner, and even before it is accomplished you may be moving to the next platform.

Why do you have to port everything forward to the newest platform? Well, presumably it is better engineered. The newest platform is presumably what people are most familiar with. But if those were the only reasons it would be hard to justify a rewrite of working software. Often the real push comes because your systems don’t work together. It’s hard to keep templates in sync across all the platforms. Multiple logins may be required. Navigation is inconsistent and incomplete. Functionality that cross-cuts pages — comments, login status, shopping cart status, etc — isn’t universally available.

A similar conflict arises when you consider how to add new functionality to a site. For example, you may want to add a blog. Do you:

  1. Use the best blogging software available?
  2. Use something native to your platform?
  3. Write something yourself?

The answer is probably 2 or 3, because it would be too hard to integrate something foreign to your platform. This form of choice means that every platform has some kind of "blog", but the users of that blog are likely to only be a subset of the users of the parent platform. This makes it difficult for winners to emerge, or for a well-developed piece of software to really be successful. Platform-based software is limited by the adoption of the platform.

Not all software has a platform. These tend to be the most successful web applications, things like Trac, WordPress, etc.

"Aha!" you think "I’ll just use those best-of-breed applications!" But no! Those applications themselves turn into platforms. WordPress is practically a CMS. Trac too. Extensible applications, if successful, become their own platform. This is not to place blame, they aren’t necessarily any worse than any other platform, just an acknowledgment that this move to platform can happen anywhere.

Beyond Platforms, or A Better Platform

One of the major goals of Deliverance is to move beyond platforms. It is an integration tool, to allow applications from different frameworks or languages to be integrated gracefully.

There are only a few core reasons that people use platforms:

  1. A common look-and-feel across the site.
  2. Cohesive navigation.
  3. Indexing of the entire site.
  4. Shared authentication and user accounts.
  5. Cross-cutting functionality (e.g., commenting).

Deliverance specifically addresses 1, providing a common look-and-feel across a site. It can provide some help with 2, by allowing navigation to be more centrally managed, without relying purely on per-application navigation (though per-application navigation is still essential to navigating the individual applications). 3, 4, and 5 are not addressed by Deliverance (at least not yet).

Deliverance applies a common theme across all the applications in your site. It’s basic unit of abstraction is HTML. It doesn’t use a particular templating language. It doesn’t know what an object is. HTML is something every web application produces. Deliverance’s means of communication is HTTP. It doesn’t call functions or create request objects [*]. Again, everything speaks HTTP.

Deliverance also allows you to include output from multiple locations. In all cases there’s the theme, a plain HTML page, and the content, whatever the underlying application returns. You can also include output from other parts of the site, most commonly navigation content that you can manage separately. All of these pieces can be dynamic — again, Deliverance only cares about HTML and HTTP, it doesn’t worry about what produces the response.

This is all very similar to systems built on XSLT transforms, except without the XSLT [†], and without XML. Strictly speaking you can apply XSLT to any parseable markup, even HTML, but the most common (or at least most talked about) way to apply XSLT is using "semantic" XML output that is transformed into HTML. Deliverance does not try to understand the semantics of applications, and instead expects them to provide appropriate presentation of whatever semantics the underlying application possesses. Presentation is more universal than semantics.

While Deliverance does its best to work with applications as-they-exist, without making particular demands on those applications, it is not perfect. Conflicting CSS can be a serious problem. Some applications don’t have very good structure to work with. You can’t generate any content in Deliverance, you can only manipulate existing content, and often that means finding new ways to generate content, or making sure you have a place to store your content (as in the case of navigation). This is why arguably Deliverance does not remove the need for a platform, but is just its own platform. In so far as this is true, Deliverance tries to be a better platform, where "better" is "more universal" rather than "more powerful". Most templating systems are more powerful than Deliverance transformations. It can be useful to have access to the underlying objects used to procude the markup. But Deliverance doesn’t give you these things, because it only implements things that can be applied to any source of content. Static files are entirely workable in Deliverance, just as any application written in Python, PHP, or even an application hosted on an entirely separate service is usable through Deliverance.

The Missing Parts

As mentioned before, two important benefits of a platform are missing from Deliverance. I’ll try to describe what I believe are the essential aspects. I hope at some time that Deliverance or some complementary application will be able to satisfy these needs. Also, I suggest some lines of development that might be easier than others.

Indexing The Entire Site

Typically each application has a notion of what all the interesting pages in that application are. Most applications have a set of uninteresting pages, or transient pages. A search result is transient, as an example. An application also knows when new pages appear, and when other pages disappear. A site-wide index of these pages would allow things like site maps, cross-application search, and cross-application reporting to be done.

An interesting exception to the knowledge an application has of itself: search results are generally boring. But a search result based on a category might still be interesting. The difference between a "search" and a "report" is largely in the eye of the beholder. An important feature is that the application shouldn’t be the sole entity allowed to mark interesting pages. Manually-managed lists of resources that may point to specific applications can allow people to usefully and easily tweak the site. Ideally even fully external resources could be included, such as a resource on an entirely different site.

To do indexing you need both events (to signal the creation, update, or deletion of an entity/page), and a list of entities (so the index can be completely regenerated). A simple way of giving a list of entities would be the Google Site Map XML resource. Signaling events is much more complex, so I won’t go into it in any greater depth here, but we’re working on a product called Cabochon to handle events.

One thing that indexing can provide is a way to use microformats. Right now microformats are interesting, but for most sites they are largely useless. You can mark up your content, but no one will do anything interesting with that markup. If you could easily code up an indexer that could keep up-to-date on all the content on your site, you could produce interesting results like cross-application mapping.

Shared Authentication And User Accounts

Authentication is one of the most common and annoying integration tasks when crossing platform boundaries. Systems like Open ID offer the ability to unify cross-site authentication, but they don’t actually solve the problem of a single site with multiple applications.

There is a basic protocol in HTTP for authentication, one that is workable for a system like Deliverance, and there are already several existing products (like repoze.who) that work this way. It works like this:

  • The logged-in username is sent in some header, e.g., X-Remote-User. Some kind of signing is necessary to really trust this header (Deliverance could filter out that header in incoming requests, but if you removed Deliverance from the stack you’d have a security hole).
  • If the user isn’t logged in, and the application wants them to log in, the application response with a 401 Unauthorized response. It is supposed to set the WWW-Authenticate header, probably to some value indicating that the intermediary should determine the authentication type. In some cases a kind of HTTP authentication is required (typically Basic or Digest) because cookie-based logins are too stateful (e.g., in APIs, or for WebDAV access).
  • The intermediary catches the 401 and initiates the login process. This might mean a redirect to a login page, and setting a cookie on successful login. The login page and setting the cookie could potentially be done by an application outside of the intermediary; the intermediary only has to do the appropriate redirects and setting of headers.
  • In the case when a user is logged in but isn’t permitted, the application simply sends a 403 Forbidden response. The intermediary shouldn’t actually do anything in this case (though maybe it could usefully add a logout link to that message). I only mention this because some systems use 401 for Forbidden, which causes no end of problems.

While some applications allow for this kind of authentication scheme, many do not. However, the scheme is general enough that I think it is justifiable that applications could be patched to work like this.

This handles shared authentication, but the only information handed around is a username. Information about the user — the real name, email, homepage, permission roles, etc — are not shared in this model.

You could add something like an internal location to the username. E.g.: X-Remote-User: bob; info_url=http://mysite.com/users/bob.xml. It would be the application’s responsibility to make a subrequest to fetch that information. This can be somewhat inefficient, though with appropriate caching perhaps it would be fine. But many applications want very much to have a complete record of all users. Changing this is likely to be much harder than changing the authentication scheme. A more feasible system might be something on the order of what is described in Indexing the Entire Site: provide a complete listing of the site as well as events when users are created, updated, or deleted, and allow applications to maintain their own private but synced databases of users.

A common permission system is another level of integration. One way of handling this would be if applications had a published set of actions that could be performed, and the person integrating the application could map actions to roles/groups on the system.

Cross-cutting Functionality

This item requires a bit of explanation. This is functionality that cuts across multiple parts of the site. An example might be comments, where you want a commenting system to be applicable to a variety of entities (though probably not all entities). Or you might want page-update notification, or to provide a feed of changes to the entity.

You might also want to include some request logger like Google Analytics to all pages, but this is already handled well by Deliverance theming. Deliverance’s aggregation handles universal content well, but it doesn’t handle content (or subrequests) that should only be present in a portion of pages.

One possible way to address this is transclusion, where a page can specifically request some other resource to be included in the page. A simple subrequest could accomplish this, but many applications make it relatively easy to include some extra markup (e.g., by editing their templates) but not so easy to do something like a subrequest. We’ve written a product Transcluder to use an HTML format to indicate transclusion.

It’s also possible using Deliverance that you could implement this functionality without any application modification, though it means added configuration — an application written to be inserted into a page via Deliverance, and a Deliverance rule that plugs everything together (but if written incorrectly would have to be debugged).

Other Conventions

In addition to this, other platform-like conventions would make the life of the integrator much easier.

Template Customization

While Deliverance handles the look-and-feel of a page, it leaves the inner chunk of content to the application. If you want to tweak something small you will still need to customize the template of the application.

It would be wonderful if applications could report on what files were used in the construction of a request, and used a common search path so you could easily override those files.

Backups and Other Maintenance

Process management can be handled by something like Supervisor, and maybe in the future Deliverance will even embed Supervisor.

But even then, regular backups of the system are important. Typically each application has its own way of producing a backup. Conventions for producing backups would be ideal. Additional conventions for restoring backups would be even better.

Many systems also require periodic maintenance — compacting databases, checking for any integrity problems, etc. Some unified cron-like system might be handy, though it’s also workable for applications to handle this internally in whatever ad hoc way seems appropriate.

Common Error Reporting

With a system where one of many components can fail, it’s important to keep track of these problems. If errors just end up in one of 10 log files, it’s unlikely anyone is closely tracking them.

One product we’re working on to help with this is ErrorEater, which works along with Supervisor. Applications have to be modified to emit errors in a specific format that Supervisor understands, but this is generally not too difficult.

Farming

Application farming is when one instance of an application can support many "sites". These might be sites with their own domains, or just distinct projects. Examples are Trac, which supports multiple projects in one instance, or WordPress MU which supports many WordPress instances running off a single database and code base.

It would be nice if you could add a simple header to a request, like X-Project-Name: foo and that would be used by all these products to select the site (or sub-site or project or any other organization unit). Then mapping domain names, paths, or other aspects of a request to the project could be handled once and the applications could all consistently consume it.

(Internally for openplans.org we’re using X-OpenPlans-Project and custom patches to several projects to support this, but it’s all ad hoc.)

Footnotes

[*] This isn’t entirely true, Deliverance internally uses WSGI which is a Python-level abstraction of HTTP calls.
[†] At different times in the past, in an experimental branch right now, and potentially integrated in the future, Deliverance has been compiled down to XSLT rules. So Deliverance could be seen even as an simple transformation language that compiles down to XSLT.

by Ian Bicking at October 07, 2008 02:56 AM

Techcrunch

Sugar High: Sugar Inc Launches API And IMDB For Fashion

Sugar Inc, the company behind a network of popular women-focused blogs that includes PopSugar, is announcing two major releases tonight that take advantage of its fashion-hungry userbase. The first, dubbed PopSugar’s CelebStyle, is positioning itself as an IMDB for style, analyzing outfits from many of television’s most popular shows. Sugar Inc is also announcing a new ShopStyle API, which gives developers access to the site’s massive database of clothing and accessories which can be used in any variety of applications. Here’s a breakdown of both announcements:

ShopStyle API
ShopStyle, which Sugar aqcuired a year ago, is a search engine for fashion. The site indexes clothing and accessories across thousands of brands and stores, presenting them to users in a uniformly formatted grid of thumbnails, prices, and descriptions.

Sugar had previously made this data available to partner sites like In Style and People, and is now opening it up to developers. CEO Brian Sugar says that apps using the API could easily be deployed across a variety of platforms, including webpages (like SaleHabit, which Sugar developed in a weekend), Facebook, and the iPhone.

In thirty days, the platform will also allow developers to convert clicks from the fashion items they’ve displayed into cash. Sugar says the revenue split will vary by app, and will be determined by the success rate of the app’s lead generation.

Fashion may not be most developers’ cup of tea, but the market for this kind of application is huge - expect to see a variety of fashion-based iPhone and web applications hit the market in the next few months.

PopSugar’s CelebStyle

PopSugar’s CelebStyle will allow users to see exactly what their favorite stars have been wearing on some of the most popular shows on TV. The site is edited by a number of Sugar employees, who will pick some of the most prominent outfits from these shows and offer links to each accessory and piece of clothing. To build and help populate the site, Sugar is leveraging the technology and partnerships behind StarBrand, which it acquired in May. Through some of these partnerships (and PR representatives), Sugar editors will have access to lists detailing exactly what the stars were wearing, so users will be assured that the items presented are authentic.

Visitors to CelebStyle will also be able to craft their own outfits using Sugar’s database of clothing, which can be added as comments beneath any article. For instance, users could put together an outfit closely mirroring a fancy get-up shown in the latest episode of Gossip Girl, but with clothes that were only a fraction of the cost.

Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.

by Jason Kincaid at October 07, 2008 01:51 AM

Techcrunch

Fight Spam With A Direct Message To Twitter

There doesn’t appear to have been an official announcement, but Twitter has begun soliciting spam reports to a “spam” user account via direct messages.

Are you a victim of Twitter spam? Just begin following @spam and send it a direct message with the username of your spammer. As the following email autoresponse to spam reporters instructs, you can send these direct messages from your mobile phone or opt for a public tweet as well:

Howdy!

Thanks for reporting spam- we’re working really hard on getting rid of it! Did you know: you can now easily report spam directly from your Twitter account? Visit:

http://twitter.com/spam

and follow the account. You can then send:

* a direct message to @spam: @moneybagsnow is a spammer!
* a direct message from your phone using d+ username + message: d spam @carmoney, @cashnow is spam!
* a reply to @spam like so: @spam this is a spam account: @bigmoney5

and we’ll take care of it. You can send as many spam user names as will fit in one direct message or @reply as long as they are designated like this: @crystal.

Note: it’s better to send a direct message over an @reply. Direct messaging keeps @replies reporting spam out of your followers’ time lines. Sending direct messages also keeps the spam account’s user name out of all search results. Because the message is private, you prevent them from benefitting from publicity. Thanks again for helping us track down spammers!

Twitter Support Team

So far 213 members have begun following @spam, which oddly has decided to follow 179 members itself. It will have to gain a much larger following to make a dent in Twitter’s spam problem. The Twitter Blacklist, a website that tracked all banned Twitter accounts before its owner lost faith in the service, lists 561 blacklisted users as of July 12th.

Read more about Twitter’s spam efforts here.

[Thanks for the tip, Rafa]

Crunch Network: CrunchBase the free database of technology companies, people, and investors

by Mark Hendrickson at October 07, 2008 01:47 AM

Techcrunch

Geode: The New Geo-Tagging Project From Mozilla Labs

Tomorrow Mozilla will launch a new geotagging project called Geode into Mozilla Labs that promises to leverage your physical location to enhance your overall browsing experience. More details will be provided in an official post tomorrow, but this is what we know already:

Geode is a Firefox add-on that understands location, enabling enriched, personalized, and localized content.

For example with Geode, a user who is looking for restaurants while they are out of town will be able load up their favorite review site and find suggestions a couple blocks away and plot directions there.

It’s unclear how Firefox actually intends to determine a user’s location, especially since Mozilla doesn’t have a mobile browser that could provide GPS data. How web apps are supposed to leverage the plugin is also yet to be seen.

Geode joins the recently announced Labs projects Ubiquity and Snowl. We’ll post more details as soon as we get them.

Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.

by Henry Work at October 07, 2008 12:32 AM

October 06, 2008

Techcrunch

Forget The Blackberry “Application Center.” The BerryStore Will Have Better Apps.

As Research in Motion prepares to open its Blackberry Application Center to answer the iPhone’s App Store, an unaffiliated startup called the BerryStore has already launched a competing app store for Blackberry Apps. What makes it better than the official BlackBerry App Center (besides the name), is that apps in the BerryStore work across both old and new BlackBerries alike (not just the upcoming BlackBerry Storm), and across carriers. The BlackBerry App center, in contrast, is designed to be a carrier-specific store, with different apps for different carriers.

AYou can download the BerryStore as an app itself by visiting www.berrystore.com on your BlackBerry. (The App Center will require users to download apps through their Blackberry browsers, which is not the best experience). Already there are about 40 apps in the store, ranging from Loopt, 3Jam, and TwitterBerry to Obopay, Citysense, and Google Mobile. All of them are currently free, although the company plans on offering paid apps in the future. Developers can get more details about how to submit apps or the BerryStore here.

Below is a list of each app currently in the BerryStore with a short description:

Books & Reference
NeoReader: Turns your Blackberry into a barcode scanner.
Blackberry Wiki: Wikipedia reader.
Beyond411: Yellow pages, maps, and directions.
MobipocketReader: Mobile e-reader.

Business & Finance
Obopay: P2P payments.
Bank of America: Manage your dwindling bank account.
NyTimes DealBook: A bookmark icon to the popular blog.
E-Trade Mobile Pro: Manage your dwindling stock portfolio.

Lifestyle
Google Mobile: Search, Maps, Gmail.
Opera: Opera Mini Web browser.
Zumobi:Mobile widgets.
Google Mail: As in Gmail.
Poynt: Local search.
Maufait InstaFind: Al-in-one 411, flight tracker, movie showtimes, stock quotes, weather, news.
Puretracks:Mobile music store.
Tellme: Voice-enabled GPS info.
reQall: Voice-to-text recorder, to-do list, and idea manager.
Nobex Radio Companion: Shows you the name of the songs playing on the radio.

News & Weather
Viigo: News, sports, entertainment, weather, stock and traffic alerts.
New York Times: Bookmark icon.
ABC News: Bookmark icon.
The Washington Post: Bookmark icon.
CNBC Mobile: Bookmark icon.
USA Today: Bookmark icon.
Slate: Bookmark icon.
PinStack.com: Forums

Social Networking
TwitterBerry: Mobile Twitter client that avoids SMS charges.
3jam: Group text messages.
eBuddy: Instant messaging app
Dexrex: Archives your text messages.
Pinger: Voice IM.

Sports
Sports Illustrated: Bookmark icon.
ESPN Mobile: Bookmark icon.

Travel & Navigation
Google Maps: What it sounds like.
GPSed: Map your GPS tracks, save them, and share them. Also geotags your photos.
Citysense: Live hotspot tracking.
WorldMate Live: A personal digital assistant for travelers

Utilities
Box.net: Access and share files on your BB.
AutoLock: Locks the keyboard.
MidpSSH: Connect to remote servers.
MiniMoni: Monitor IP traffic.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

by Erick Schonfeld at October 06, 2008 11:26 PM

The Scobleizer

Scobleizer


Are you depressed about the market? Well, just think of all that we have that costs nothing!

Here, Mitch Joel put together a whole conference for you. Costs nothing. Just grab a glass of wine and watch.

It reminded me of how much I love TED Talks. Watch the video of famous conductor Benjamin Zander, it’ll get your mood lifted.

After that, why don’t you visit the Library of Congress to see how they are saving our old photos and putting them onto Flickr for you to peruse. Don’t think social media matters to something as stodgy as the Library of Congress? Think again!

Our stocks might be worthless, but at least we have good free stuff to watch on the Internet.

That’s the “cup is half full” approach to life.

      

by Robert Scoble at October 06, 2008 10:58 PM

Bruce Eckel

Photos & Article from Info Magazine (Brazillian)

While I was in Brazil, I was interviewed for a Brazillian technical magazine, and the issue was just published.

October 06, 2008 10:01 PM

Bob Ippolito

Mochi Media hiring for Python web developer job in San Francisco

I don't often post about open jobs at Mochi Media on my blog, but it seems that most of the awesome people we have here found out about Mochi Media from me so I might as well ;)

Web development at Mochi Media happens primarily in Python and Pylons with Genshi templates. We're currently using mostly MochiKit for the JavaScript heavy lifting and we talk to all of our backend services (which are either Python or Erlang) via JSON. We're using a bunch of other cool technologies such as memcached, nginx, PostgreSQL, etc. We also often contribute a lot of what we do back to the open source community (e.g. MochiKit, MochiWeb, simplejson, etc.) and would love adding some more team members that were interested in helping out with our open source efforts too! We don't require experience with all of these frameworks and tools, we just need smart people that have a really solid understanding of web development in Python.

Mochi Media runs MochiAds, a monetization platform for Flash games, and MochiBot, an analytics service for Flash content... so we've got tons and tons of data to work with and very interesting scale problems to address. Our primary service, MochiAds, is a monetization platform for Flash games... so part of the responsibilities of being a Mochi employee is to approve all the latest in Flash games for paid advertisements from our network ;) There's even an internal competition to see who approves the most games each week! You can check some of them out at http://www.mochiads.com/games/

Mochi Media was founded in 2005 by Jameson Hsu and myself, and we're backed by Accel Partners and Shasta Ventures. We've got a great team and are always looking to add the best people we can find. If you kick ass at Python and love building cool apps for the web, talk to us!

This position is full-time and on-site in San Francisco, CA. We're very easily accessible by BART and Caltrain in SOMA, at 2nd st. and Mission st. More evidence that Mochi Media is a cool place to work can be found on the mochimedia Flickr photostream: http://flickr.com/photos/mochimedia

More info and application instructions are here: http://www.mochimedia.com/about/careers/#webengineer

by bob at October 06, 2008 09:57 PM

Techcrunch

What Does the Slow-Down Mean for Gadget Lovers?

With news of doom and gloom coming at us from every side I would like to take a moment to look at the market as it stands and assess what a slow-down/recession will mean for us, the gadget obsessed. To recap, we are in this mess because of what amounts to a credit freeze. Companies that once could request "cash" to cover overnight expenses - we're talking millions and billions of dollars worth of cash - are now told they can only withdraw a significant percentage less than they were previously allowed to access. This, in turn, reduces capital expenditures and slows hiring - if you can't get a quick loan you can't cover a new factory or a make a bigger payroll. This, in turn, slows large purchases and raises unemployment which, in turn, makes us all freak out.

by John Biggs at October 06, 2008 09:52 PM

Techcrunch

Apple Met Goal of 10 million iPhones Sold in 2008

By using some fairly interesting IMEI collection, the folks at Mac Observer have found that Apple sold 10 million iPhones in 2008, reaching and potentially surpassing Steve's original stated goal. By looking at phones sold over the last few months, Mac Observer's "Apple Finance Board" found that the phone has gone through nine 1 million unit runs. Adding this to the known sales they found the total number was far above analyst expectations.

by John Biggs at October 06, 2008 09:51 PM

The Google Blog

New Technology Roundtable series

We've just posted the first three videos in the Google Technology Roundtable Series. Each one is a discussion with senior Google researchers and technologists about one of our most significant achievements. We use a talk show format, where I lead a discussion on the technology.

While the videos are intended for a reasonably technical audience, I think they may be interesting to many as an overview of the key challenges and ideas underlying Google's systems. And of course they offer a glimpse into the people behind Google.

The first one we made is "Large-Scale Search System Infrastructure and Search Quality." I interview Google Fellows Jeff Dean and Amit Singhal on their insights in how search works at Google.

The next title is "Map Reduce," a discussion of this key technology (first, at Google, and now having a great impact across the field) for harnessing parallelism provided by very large-scale clusters computers, while mitigating the component failures that inevitably occur in such big systems. My discussion is with four of our Map Reduce expert engineers: Sanjay Ghemawat and Jeff Dean again, plus Software Engineers Jerry Zhao and Matt Austern who discuss the origin, evolution and future of Map Reduce. By the way, this type of infrastructure underlies the infrastructure concepts in our recent post on "The Intelligent Cloud."

The third video, "Applications of Human Language Technology," is a discussion of our enormous progress in large-scale automated translation of languages and speech recognition. Both of these technology domains are coming of age with capabilities that will truly impact what we expect of computers on a day-to-day basis. I discuss these technologies with human language technology experts Franz Josef Och, an expert in the automated translation of languages, and Mike Cohen, an expert in speech processing.

We hope to produce more of these, so please leave feedback at YouTube (in the comments field for each video), and we will incorporate your ideas into our future efforts.

[Cross-posted on the Google Research Blog.]

Posted by Alfred Spector, VP of Research and Special Initiatives

by Karen (noreply@blogger.com) at October 06, 2008 09:47 PM

Techcrunch

Meet Mark Zuckerberg In Munich, Tomorrow

Attention all TechCrunch readers in Munich: here’s your chance to meet Facebook founder Mark Zuckerberg. (Hmm, I wonder if he’ll be making some sort of announcement). He will be giving a speech at a closed, invite-only event on Tuesday evening, October 7 at Amiando headquarters. Amiando CEO Felix Haas is offering to give admission to one lucky TechCrunch reader. (That’s right, only one). Explains Haas in an e-mail:

Mark will give a speech to the selected guests about Facebook’s plans for Europe and Germany. After the speeches and an extensive Q&A session with Mark Zuckerberg there will be plenty of time to get-together with Mark at dinner & drink. I want to offer admission for one of your readers.

All you have to do to apply is explain why you want to meet Zuckerberg in comments. The person with the best reason gets a ticket. (Put in your real e-mail address so Felix can contact you in case you win).

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

by Erick Schonfeld at October 06, 2008 09:44 PM

Bob Ippolito

simplejson 2.0.2

simplejson (documentation) is a simple, fast, complete, correct and extensible JSON (RFC 4627) encoder/decoder for Python 2.3+. It is pure Python code with no dependencies, but features an optional C extension for speed-ups.

simplejson 2.0.2 is a minor update:

  • Fixes MSVC2003 build regression
  • Fixes Python 2.4 compatibility in _speedups.c

by bob at October 06, 2008 09:30 PM

Techcrunch

SimplyBox: Drag-And-Drop Clips Of The Web For Research, Sharing

Web bookmarks are badly in need of an overhaul. Sure, they’re fine for jumping to the dozen or so sites you visit on a daily basis, but they quickly become unwieldy whenever they’re used for research purposes. Folders of bookmarked job listings or apartment openings on Craigslists become practically useless as you frequently need to revisit them to remind yourself why you bookmarked them in the first place.

Tonight sees the launch of SimplyBox, a San Francisco-based startup that allows users to clip portions of websites in a visual way, overcoming some of the shortcomings of traditional bookmarks. SimplyBox is currently available as plugins for Internet Explorer and Firefox, with a Safari version on the way. The site will go live at 6 PM PST tonight.

To use SimplyBox, you hit the “Box and Save” button in the browser toolbar, which turns your mouse arrow into a targeting cursor and displays a series of “boxes” at the bottom of the screen. After highlighting the portion of the page you’d like to save for later, you drag it into the box you’d like to store it in. The whole process is very intuitive and only takes a few seconds.

Clicking on the toolbar’s SimplyBox logo brings you to your profile page, where you can access and sort through each of your boxes. Clippings appear as large images that clearly show their contents, and can be displayed in a desktop view (where you can drag them around), a list view, or a grid. Users can also add comments to each of the snippets.

In practice, the plugin seems to work well, especially when conducting research across a number of sites. For example, house hunters could easily use the service to keep track of the homes they’re interested in, clipping photos and summaries for future reference. The site also allows users to share their boxes with friends, so they collaborate on a single project. Finally, the toolbar includes a “box and send” function that sends snippets in Emails as image files.

SimplyBox’s biggest flaw is that its snippets are simple images - they don’t retain any of the functionality of their original source. If you were to snap a clip of a YouTube video, it would only appear as an image, and you’d have to visit the linked page to see the video itself. That said, it’s still a big improvement over simple bookmarks.

There are a number of other “clipping” tools available, including Sazell, Diigo, and to some extent, Safari’s Webclip.

Crunch Network: CrunchBase the free database of technology companies, people, and investors

by Jason Kincaid at October 06, 2008 09:25 PM

Techcrunch

GameFly Goes Mobile

GameFly, which is perhaps most easily explained as the NetFlix of the gaming world, today launched a new version of their site specifically optimized for mobile handsets.

Through the magic of user agent detection, getting to the mobile site is a one step process: just type the standard GameFly.com address into the browser of any mobile handset. Once there, users can search through the GameFly library, order games, switch up their "GameQ" queue, read reviews, and manage their account.

by Greg Kumparak at October 06, 2008 09:14 PM

IronPython URLs

Compiling the DLR for Silverlight and IronPython with Silverlight RC0

Silverlight Release Candidate 0 is out, this means that the old versions of IronPython and the Dynamic Language Runtime don't work with the new version.

Silverlight includes a cut down version of the .NET framework called the Core-CLR. IronPython and DLR have to be compiled against the Core-CLR assemblies for the version of Silverlight you are targeting.

But IronPython is Open Source, so you can just compile it against the new Silverlight assemblies right? Jimmy Schementi explains:
"Of course! Everything should just work, since there were no major breaking changes in Silverlight that affect the DLR between Beta2 and RC0. So, you hacked up the csproj files to point at mscorlib.dll, system.dll, etc in the new Silverlight install directory (C:\Program Files\Microsoft Silverlight\2.0.30923.0), compile, and it builds fine. Then you try to run an app ..."

Next you need to make a simple change to your dynamic application to tell it about the new version of the runtime. Jimmy covers this in his blog entry as well. Of course rather than compiling IronPython yourself, it is easier to wait for Jimmy to release a new version of the SDK.
Jimmy also explains the changes in the DLR hosting API if you are hosting IronPython or IronRuby in your Silverlight application.

by Fuzzyman (noreply@blogger.com) at October 06, 2008 09:06 PM

Techcrunch

Granting Credit On The Fly. Is Bill Me Later Part Of the Problem Or Part Of The Solution?

billme-later-logo.png

On the same day the public markets are tanking because of the spreading credit crisis, we see one of the biggest M&A exits of the year with eBay acquiring Bill Me Later for $945 million ($820 million in cash, plus an extra $125 million in options). The only other tech exits of this size in 2008 were Sun buying MYSQL for $1 billion (which involved less cash and more options), AOL buying Bebo for $850 million.  This is for a company that lets consumers defer payment when they buy things online.  Remember, loose credit is part of the reason we are in the current economic mess.

So is Bill Me Later part of the problem or part of the solution? I put that question to Michael Kwatinetz, the former Wall Street tech analyst who is now a partner at Azure Capital, the biggest shareholder in Bill Me Later.  He explained to me how Bill Me Later works, and how it actually has more stringent credit controls than most credit cards:

The problem is people who can’t afford to pay for things are financing things. If you have the proper controls, you don’t allow that to happen.

We don’t grant credit limits. We grant credit on a transaction basis. If you are somebody who is not paying us, or running up your bills in other places, we don’t grant credit.

Traditional credit cards, in contrast, let you run up your bill up to a pre-determined credit limit.  With each transaction, BillMeLater check your credit score, credit outsanding, status with credit agencies, and a few other criteria.  And it either approves your credit or it doesn’t for each purchase in less than three seconds.  Kwatinetz says that the company tightened its lending policies about a year ago, and claims that the nonpayment rate is “probably the lowest of anyone on the Web.”

The value of the company lies in its algorithms that determine credit risk and in its ability to use the Internet as a platform for granting credit.  All of the data that its algorithms consider is pulled in via Web APIs from credit agencies and other data providers.

For eBay, Bill Me Later will help expand its Paypal franchise to consumer credit.  Bill Me Later is already used all over the Web by more than four million customers at places like Amazon, the Apple Store, JetBlue, and Walmart,com.  This year the company is expected to finance $1 billion worth of online purchases and bring in $125 million in revenues.  Revenues are estimated to be $150 million in 2009 (a 20 percent growth rate), but the acquisition will also be dilutive to eBay’s earnings, meaning the business is not yet profitable on a net income basis.

Still, you can see the appeal of an all-online credit service.  Traditional credit card companies pay about $150 for each customer who signs up for a new Visa or Mastercard, and it usually takes a coupel weeks to process each account through the mail.  Bill Me Later acquires customers pretty much instantly at the time of purchase for less than 10 percent of what it costs the credit card companies.

Announcing a large acquisition like this that is essentially a foray into the troubled credit markets may seem counterintuitive, but after a string of deals that didn’t make much sense for eBay (Skype, StumbleUpon), this could end up being one of its smartest non-core acquisitions since Paypal.

CrunchBase Information
Bill Me Later
Information provided by CrunchBase

Crunch Network: CrunchBoard because it’s time for you to find a new Job2.0

by Erick Schonfeld at October 06, 2008 08:56 PM

The Scobleizer

Scobleizer


The DOW is down about 600 about 700 about 370 right now. Yahoo Finance has the bad news. UPDATE: what a bumpy ride. At one point it was down 748 on my screen and ended up down “only” (Yahoo Finance’s headline) 370.

Here’s the deal: many of us are afraid. That fear is changing our behavior (I’m hoarding, aren’t you?). That causes the market to go down.

How does the market stop going down? The death spiral will have to hit the bottom of our fears.

Where do you see the market hitting bottom? I think we’ll test 8,000 by end of the year (some people say probably by the end of the week if it keeps going down line this).

As they say, we have only to fear fear itself. Hope you’re doing OK. This is looking like one heck of a nasty storm on the horizon, isn’t it? 2009 is looking like a pretty tough year.

Anyone have any positive news to get us off of our fears?

      

by Robert Scoble at October 06, 2008 06:06 PM

Techcrunch

SearchMe Calls Out Yahoo On BOSS Screenshot

SearchMe, a new Sequoia-backed search engine that launched in March, isn’t happy with some of the screen shots that have been seen around the Internet promoting Yahoo’s new BOSS (“Build Your Own Search Service”) product.

In particular, this shot, which was included in a CNET article, shows what appears to be SearchMe’s search interface (itself borrowed from Apple’s iTunes album browser) with the logo cut off (compare it to this). CNET describes the screen shot as “One idea Yahoo showed for BOSS: show miniature versions of the Web pages returned by search results.”

To be fair to Yahoo, the screen shot was included in a press briefing document (the slide, which I also received, is shown below) and Yahoo was using it only as an example to show how the service worked. Yahoo has also said that they reached out to SearchMe to discuss a partnership, as they’ve done with scores of other companies. In the briefing, Yahoo didn’t specifically call attention to the slide, and it was made clear to journalists (or at least me) that the product wasn’t made for front end design, just as a search web service.

Yahoo probably should have left the SearchMe logo on the screen shot, but given how many other things there are to criticize Yahoo for these days, I’m inclined to give it a pass.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

by Michael Arrington at October 06, 2008 05:41 PM

Techcrunch

RIM to Launch BlackBerry “Application Center”

appcenter You're looking at the BlackBerry Application Center, RIM's answer to Apple's App Store and Google's Android Market. Same basic concept as the other two—you browse and buy applications that enhance your BlackBerry experience. The Application Center—we're running out of variants of the term “application store!”—is set to debut with the BlackBerry Storm software version 4.7. All application data will be stored at the carriers' locale; RIM is totally out of the loop as far as that goes. It's supposed to differentiate the Application Center from the App Store in that regard—carriers can put the applications they want on their own little store.

by John Biggs at October 06, 2008 04:48 PM

Voidspace Del.icio.us Links

Neil Gaiman

Techcrunch

AOL-Yahoo Merger Details Emerge; Deal Could Happen This Month

twoheaded.pngYahoo is continuing its marathon merger discussions with AOL, sources close to the negotiations have whispered to us, and a deal could happen as early as this month. Is this just a rehash of the reported discussions in February and then again in April?

Yes and no. It’s clear that AOL’s parent company, Time Warner, wants this deal more than ever. What isn’t clear is whether AOL’s assets will fix any of Yahoo’s problems.

The deal structure that is currently being discussed is Yahoo’s acquisition of AOL (content, services and advertising), minus their subscription dial up business. That plus a couple of billion dollars in cash from Time Warner gets them approximately a third of the combined entity. Time Warner’s AOL headache is gone, and they have a stake in the world’s most valuable chess piece in the Google/Microsoft search and advertising war.

Factors favoring a deal: the companies believe Yahoo’s advertising platform would monetize AOL assets far beyond what they’re generating today (a little over $2.4 billion annually). And those against: combined dominance in mail (they’d have 48% of all worldwide email accounts according to Comscore, with Microsoft #2 at 42%) and instant messaging (39% worldwide combined market share, compared to 55% for Microsoft). In reality, though, email and instant messaging market share are only a problem if Microsoft then comes in and buys the combined entity.

Yahoo gets to make a case to stockholders that they dominate the online portal/services/content world, and who cares if they outsource search advertising to Google. Our position is that they can’t succeed in the long run without strong and competitive search advertising, although it may take the Department of Justice to get that message through to Yahoo’s executive team. Even after these entities combine, if they do, Yahoo still has a major long term competitive problem on its hands.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

by Michael Arrington at October 06, 2008 04:16 PM

A Softer World

Techcrunch

Tech Stocks Take It On The Chin As Market Continues To Freefall

It’s a blood bath out there this morning. The S&P 500 is at a four-year low as the credit crisis keeps getting worse, despite the passage of the government’s $700 billion bailout plan. The market is taking tech stocks down with it. Google is down 4 percent to $368, its lowest point since 2006. Apple is down 6 percent to $91. Microsoft is down nearly 5 percent to $25. Amazon, Yahoo, eBay—all down.

Fears of a credit freeze are growing as the contagion spread to banks in Europe. The Fed is already flooding the market with more cash through new powers it was granted in the bailout package. All of this makes you wonder if A) the U.S. government acted fast enough and B) whether the bailout package is going to end up doing any good.

As far as tech stocks are concerned, already as I write this, there seems to be somewhat of a rally going on in some of these stocks (particularly Google) from the lows where they opened. But if the economy falters, tech stocks won’t be a safe haven for investors, even if they are cash-rich and not as exposed to the credit debacle as companies in other sectors. The markets always tend to overreact to systemic risk because nobody knows how far the problems are going to spread. What we are seeing is panic in the face of the unknown. It reminds me of the market panic after 9/11. Investors whop loaded up on tech stocks then ended up making a lot of money.

Does this signal a buying opportunity, or are investors better off running for the hills? Who is buying (or selling) what out there? Tell us in comments.

Crunch Network: CrunchBoard because it’s time for you to find a new Job2.0

by Erick Schonfeld at October 06, 2008 03:01 PM

Techcrunch

Ebay Spends More Than $1.2 Billion To Buy Bill Me Later And DBA.dk, And Lays Off 10% Of Employees

It’s a big day for eBay and CEO Jon Donahoe. The company is laying off 10% of its workforce, or about 1,000 employees plus several hundred temporary positions. They will take a $70 million to $80 million restructuring charge around the layoffs, they said.

And they’ve made two acquisitions: Denmark’s DBA for $380 million and Amazon-funded Bill Me Later for $820 million in cash and approximately $125 million in outstanding options.

Regarding the two acquisitions: We covered dba.dk, Denmarks’ leading classifieds site, earlier this morning. Bill Me Later, the larger acquisition, is a service that let’s ecommerce partners issue instant credit to buyers. You enter your birth date and last four digits of your social security number online, and it does a credit check on you in three seconds to determine whether you are worth the risk. Bill Me Later pays the merchant, and sends you a bill. The company has raised a ton of cash - at least $272 million - from Amazon, Azure Capital Partners, Chase Paymentech, Crosspoint Venture Partners, First Data Corp., and others.

eBay stock is at a 52 week low, we’ll see how it does in the market today. (Update: well, the markets are tanking, so it’s not really a good day to judge eBay alone).

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

by Michael Arrington at October 06, 2008 03:00 PM

Jon Udell

jonudell


Social networks are Petri dishes in which we can watch memes emerge and spread by imitation. Three years ago, I traced the effect of a powerful one created by the ACLU: a fictional screencast about a dystopian future in which identity and privacy have gone horribly wrong. What I found when I looked at the data was that, although forward thinkers and actors in the realm of digital identity had only recently become aware of the ACLU’s powerful meme, it had been active for 18 months, most forcefully at the beginning of that span.

In that case the meme was an idea which, because it was neatly represented by an URL, could be tracked by using services like del.icio.us and bloglines as proxies for the attention that flows to an URL.

In other cases, a meme is best represented by a word — often, a neologism. There’s no canonical URL to track, but there are other ways to monitor the spread of the meme. Search engines, for example. In the case of screencast, for example, there were 200 Google hits for screencast in April 2005, 60,000 in June 2005, 325,000 in November 2005, and there are 3,000,000 today.

I’m always on the lookout for new ways to make these kinds of observations. Yesterday I encountered Pecha Kucha for the first time. It has a Wikipedia page, so the revision log there is one source of insight.

Since I encountered the phrase on Twitter, I tried a different strategy. While relaying a definition of the term, I used the tag #pechakucha. I realized that these Twitter “hashtags” are another proxy for linguistic memeflow, so I plotted occurrences of the tag on a Timeline. There were only 16 occurrences as of yesterday, so it’s a little sparse, but the same approach can be used to provide insight into the birth and evolution of any Twitter hashtag.

Here’s a Timeline for #quotes. It started on April 6, 2008, when Leonardo Souza quoth: “#quotes ‘This story, like any story worth telling, is about a girl’”, which evidently is from Spider-Man.

One of the nice features of Timeline, one of David Huynh’s many ingenious creations, is this condensed summary of activity:

Here we can see sporadic use of #quotes from April to the first of September, and then much heavier use. What happened on September 1? Tim O’Reilly, a powerful meme transmitter and amplifier, quoth: “‘The skill of writing is to create a context in which other people can think.’ Edwin Schlossberg. #quotes”

In Timeline we can watch other Twitter users immediately begin to use and transmit the #quotes meme:

This method will be most useful for watching Twitter hashtags that haven’t yet been widely adopted. If you apply it to, say, #ike you’ll run into two problems. First, Twitter’s API caps the number of search results you can retrieve, so in the case of #ike we can only see back as far as September 18. Second, Timeline struggles to display thousands of events.

These are general problems. No matter which Petri dish we observe — del.icio.us tagspace, the blogosphere, Twitter — our ability to watch memes evolve is limited by the amount of data we can gather, and also by our ability to effectively visualize what data we can gather. I expect both constraints to gradually erode. As they do, this game of meme tracking will become even more interesting.

      

by Jon Udell at October 06, 2008 02:01 PM

Techcrunch

GigaOm Ignores My Advice, Raises Another $4.5 Million

Om Malik’s GigaOm has raised another round of financing - $4.5 million - bringing the total amount the blog network has raised to $5.3 million. This round was led by Alloy Ventures. Previous investor True Ventures joined the round as well.

GigaOm, one of the first blog networks to raise money, has joined b5Media, Silicon Alley Insider, VentureBeat and Paid Content (acquired), among others, in raising capital. Holdout ReadWriteWeb is said to be testing the waters and looking for a round of funding as well. This is something I’ve recommended against, and we are one of the last large blog networks to remain independent.

Still, I can’t blame Om’s reasons for raising the round - aggressive expansion, software development and a nest egg for troubled financial times ahead. Congrats to him and his team, particularly newly promoted CEO Paul Walborsky.

Crunch Network: CrunchBase the free database of technology companies, people, and investors

by Michael Arrington at October 06, 2008 02:00 PM

Raymond Chen: The Old New Thing

Eventually, nothing is special any more

Commenter ulric suggested that two functions for obtaining the "current" window should exist, one for normal everyday use and one for "special use" when you want to interact with windows outside your process.

I'd be more at ease however if the default behaviour of the API was to return HWND for the current process only, and the apps that really need HWND from other potentially other processes would have to be forced to use another API that is specifically just for that.

This is an excellent example of suggesting something that Windows already does. The special function has become so non-special, you don't even realize any more that it's special.

Originally, in 16-bit Windows, the function for getting the "current" window was GetActiveWindow. This obtained the active window across the entire system. One of the major changes in Win32 is the asynchronous input model, wherein windows from different input queues receive separate input. That way, one program that has stopped responding to input doesn't clog up input for other unrelated windows. Win32 changed the meaning of GetActiveWindow to mean the active window from the current input queue.

In 16-bit Windows, there was only one input queue, the global one. In 32-bit Windows, each thread (or group of input-attached threads) gets its own input queue.

As a result of this finer granularity, when a program was ported from 16-bit Windows to 32-bit Windows, it didn't "see" windows from other programs when it called functions like GetFocus or GetActiveWindow. As every Win32 programmer should know, these states are local to your input queue.

Okay, let's look at what we've got now. GetFocus and GetActiveWindow give you the status of your input queue. In other words, in a single-threaded program (which, if you're coming from 16-bit Windows, is the only type of program there is), calling GetActiveWindow gives you the active window from your program. It doesn't return the active window from another program.¹ Things are exactly as ulric suggested!

Now let's look at the second half of the suggestion. If a program really needs to get a window from potentially other processes, it would have to use some other function that is specifically just for that. And indeed, that's why the GetForegroundWindow function was added. The GetForegroundWindow function is the special function specifically designed for obtaining windows from other processes.

Therefore, we did exactly what ulric recommended, and it still turned into a mess. Why?

Because once you create something special, it doesn't remain special for long.

It may take a while, but eventually people find that the regular function "doesn't work" (for various definitions of "work"), and they ask around for help. "When I call GetActiveWindow, I'm not getting the global active window; I'm just getting the local one. How do I get the global one?" Actually, they probably don't even formulate the question that clearly. It's probably more like "I want to get the active window, but GetActiveWindow doesn't work."

And then somebody responds with "Yeah, GetActiveWindow doesn't work. I've found that GetForegroundWindow works a lot better."

The response is then "Wow, that works great! Thanks!"

Eventually, the word on the street is "GetActiveWindow doesn't work. Use GetForegroundWindow instead." Soon, people are using it for everything, waxing their car, calming a colicky baby, or improving their sexual attractiveness.

What used to be a function to be used "only in those rare occasions when you really need it" has become "the go-to function that gets the job done."

In fact, the unfashionableness of the active window has reached the point that people have given up on calling it the active window at all! Instead, they call it the foreground window from the current process. It's like calling a land line a "wired cell phone".

Requiring a new flag to get the special behavior doesn't change things at all. It's the same story, just with different names for the characters. "GetFocalWindow² doesn't work unless you pass the GFW_CROSSPROCESS flag." Soon, everybody will be passing the GFW_CROSSPROCESS not because they understand what it does but just because "That's what I was told to do" and "It doesn't work if I don't pass it."

Footnotes

¹Assuming you haven't run around attaching your thread to some other program's input queue. This is a pretty safe assumption since the AttachThreadInput function didn't exist in 16-bit Windows either.

²GetFocalWindow is an imaginary function created for the purpose of the example.

[Raymond is currently away; this message was pre-recorded.]

by oldnewthing at October 06, 2008 02:00 PM

Techcrunch

Semantic Search Engine Hakia Now Says It Can Filter Results By How Credible They Are

On the Internet, nobody knows your site is a dog (to paraphrase the famous New Yorker cartoon). At least not yet. Semantic search engine Hakia wants to change that. Ask.com is not the only search engine rolling out a redesign today. So is Hakia, which is introducing tabs to its search interface. One of the tabs is “credible sites.” These are results from sites that have been vetted by librarians and information specialists (although anyone can suggest sites). So far, Hakia has built out a directory of credible sites around health, medical, and environmental issues.

The “credible” results tend to come from government, university, medical, and news sites. For instance, here are the credible results for “green buildings” and “common cold.”

The idea is if your site is a dog, it won’t get on the list. While this white-list approach could improve the quality of results, it also seems way too easy to game. Any spammer can try to get their site on the truthful and authoritative list. And they will.

Hakia’s redesign includes some other new tabs as well. Namely “news,” “images,” and “meet others.” (No, this third one is not a semantic dating service. It is a social feature that lets you join groups and discussion forums around the same topics that you are searching for). The results are all based on semantic matches within each domain. (See the screenshots below). But I am not sure how much better they are than Google’s. For instance, here is Hakia’s image search results for “global warming”. Now try the same search on Google image search. In both cases, the results are underwhelming.

CrunchBase Information
Hakia
Information provided by CrunchBase

Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.

by Erick Schonfeld at October 06, 2008 01:58 PM

The Google Blog

The VP debate: Candidates, questions, and queries

If information is the currency of democracy, as Thomas Jefferson allegedly said, then during last Thursday's vice-presidential debate between Senator Biden and Governor Palin a lot of people used Google Search to get a bit wealthier, metaphorically speaking. Using Google Hot Trends, we can see some of the more interesting things that people were researching, and you can do the same to follow along yourself during tomorrow night's second presidential debate (9 PM ET). But first, here's what people were curious about during the VP match.

Many people were simply interested in understanding the meaning of particular terms. Governor Palin called Senator McCain a "maverick" several times, sending many viewers to Google to query definition of maverick, what is a maverick, and define:maverick.

As the debaters spoke, voters queried for more information.

When Biden mentioned that the "theocracy controls the security apparatus" in Iran, users searched for the meaning of theocracy — as they did when he spoke of the windfall profits tax.

Getting these definitions got a bit tougher when the candidates couldn't even agree on pronunciation. Discussion about a certain type of energy caused a flurry of queries: nucular vs nuclear, nuclear pronunciation, palin nucular, and even nukular. And when Senator Biden talked about the "7,000 madrasses built along [the Pakistani-Afghan] border", the queries ranged from madrass, madrases, madrasa, and even madras, a major city in India that's most definitely not on the Pakistani-Afghan border.

Governor Palin's claim that "Israel is in jeopardy of course when we're dealing with Ahmadinejad as a leader of Iran" led viewers to try to learn more about this leader even if they could not spell his name. They searched for [Achmadinijad], [Akmadinijad], [Akmadinajad], and the correct Ahmadinejad. Some did not even try, instead looking for [president Iran] and [Iran leader]. The Governor also referred to General McKiernan, the U.S. military leader in Afghanistan, as "McClellan", sending viewers in search of McClellan, general in Afghanistan, General McClellan Afghanistan, and general Afghanistan surge. Some searchers eventually did find the correct general, but not that many.

Historical references abounded. When Senator Biden claimed "This is the most important election you will ever, ever have voted in, any of you, since 1932", some people wanted to know what it was about the 1932 presidential campaign between Herbert Hoover and Franklin Roosevelt that was so special. And twice as many them wanted to know about that "shining city on a hill", a phrase from Ronald Reagan's farewell address that was originally coined in 1630 by John Winthrop.

When Senator Biden offered a civics lesson ("Article I of the Constitution defines the role of the vice president of the United States, that's the Executive Branch") many people checked, and learned that Article I of the Constitution describes the legislative branch of the U.S. government. The executive branch is described in Article II. Others just searched directly for the role of vice president and vice president duties.

People searched on clean coal and took a look at Senator Biden's position (as the candidate asked them to) with queries like Biden clean coal.

These are some of the more interesting queries, but which were the most popular ones? Among the candidates, Senator Biden was a big winner. Searches on him soared more than 70-fold, compared to a week earlier. Governor Palin, much more of a search favorite in the weeks leading up to the debate, only saw a 6x jump, but her volume outpaced Senator Biden's.