Planet Voidspace

Techcrunch: Netvibes Partners With Russian Web Portal Rambler.ru

Jason Kincaid — 2008-10-07T03:12:19+00:00

Netvibes, the site that lets users customize their homepages with a variety of widgets, has partnered with Rambler.ru to bring its widgets to the massive Russian web portal. Rambler is the Yahoo of Russia, with an estimated 40 million users and 3 billion monthly pageviews. The deal is being described as “multi-year” and worth “multi-millions”, but further details haven’t been disclosed. Netvibes availability on Rambler.ru is expected to begin in November.

This marks the first time Netvibes has licensed its platform for installation and distribution to an independent third party, and probably won’t be the last. In order to stay competitive with other widget hubs like iGoogle, Netvibes would do well to spur its growth by offering its widgets to other region-specific portals (that said, Netvibes has been doing well, with a reported 500 billion widgets served montly). According to the press release, the Rambler homepage will include Google Search, Blinx video search, and a number of Russian services like Price.ru.

In July Google acquired Begun, a contextual ad service, from parent company Rambler. As part of the $140 million deal, Rambler has been using Google for some of its advertising and search functions.

Crunch Network: CrunchBoard because it’s time for you to find a new Job2.0

Ian Bicking: The Philosophy of Deliverance

Ian Bicking — 2008-10-07T02:56:27+00:00

I’ll be attending PloneConf this year again, giving a talk about Deliverance. I’ve been working on Deliverance lately for work, but the hard part about it is that it’s not obviously useful. To help explain it I wrote the philosophy of Deliverance, which I will copy here, to give you an idea of what I’ve been doing:

Why is Deliverance? Why was it made, what purpose does it serve, why should you use it, how can it change the way you do web development?

On the Subject of Platforms

Right now we live in an age of platforms. Developers (or management or coincidence) decides on a platform, and that serves as the basis for all future development. Usually there’s some old things from a previous platform (or a primordial pre-platform age: I’m looking at you formmail.pl!) The goal is always to eliminate all of these old pieces, rewriting them for the new platform. That goal is seldom attained in a timely manner, and even before it is accomplished you may be moving to the next platform.

Why do you have to port everything forward to the newest platform? Well, presumably it is better engineered. The newest platform is presumably what people are most familiar with. But if those were the only reasons it would be hard to justify a rewrite of working software. Often the real push comes because your systems don’t work together. It’s hard to keep templates in sync across all the platforms. Multiple logins may be required. Navigation is inconsistent and incomplete. Functionality that cross-cuts pages — comments, login status, shopping cart status, etc — isn’t universally available.

A similar conflict arises when you consider how to add new functionality to a site. For example, you may want to add a blog. Do you:

Use the best blogging software available?
Use something native to your platform?
Write something yourself?

The answer is probably 2 or 3, because it would be too hard to integrate something foreign to your platform. This form of choice means that every platform has some kind of "blog", but the users of that blog are likely to only be a subset of the users of the parent platform. This makes it difficult for winners to emerge, or for a well-developed piece of software to really be successful. Platform-based software is limited by the adoption of the platform.

Not all software has a platform. These tend to be the most successful web applications, things like Trac, WordPress, etc.

"Aha!" you think "I’ll just use those best-of-breed applications!" But no! Those applications themselves turn into platforms. WordPress is practically a CMS. Trac too. Extensible applications, if successful, become their own platform. This is not to place blame, they aren’t necessarily any worse than any other platform, just an acknowledgment that this move to platform can happen anywhere.

Beyond Platforms, or A Better Platform

One of the major goals of Deliverance is to move beyond platforms. It is an integration tool, to allow applications from different frameworks or languages to be integrated gracefully.

There are only a few core reasons that people use platforms:

A common look-and-feel across the site.
Cohesive navigation.
Indexing of the entire site.
Shared authentication and user accounts.
Cross-cutting functionality (e.g., commenting).

Deliverance specifically addresses 1, providing a common look-and-feel across a site. It can provide some help with 2, by allowing navigation to be more centrally managed, without relying purely on per-application navigation (though per-application navigation is still essential to navigating the individual applications). 3, 4, and 5 are not addressed by Deliverance (at least not yet).

Deliverance applies a common theme across all the applications in your site. It’s basic unit of abstraction is HTML. It doesn’t use a particular templating language. It doesn’t know what an object is. HTML is something every web application produces. Deliverance’s means of communication is HTTP. It doesn’t call functions or create request objects [*]. Again, everything speaks HTTP.

Deliverance also allows you to include output from multiple locations. In all cases there’s the theme, a plain HTML page, and the content, whatever the underlying application returns. You can also include output from other parts of the site, most commonly navigation content that you can manage separately. All of these pieces can be dynamic — again, Deliverance only cares about HTML and HTTP, it doesn’t worry about what produces the response.

This is all very similar to systems built on XSLT transforms, except without the XSLT [†], and without XML. Strictly speaking you can apply XSLT to any parseable markup, even HTML, but the most common (or at least most talked about) way to apply XSLT is using "semantic" XML output that is transformed into HTML. Deliverance does not try to understand the semantics of applications, and instead expects them to provide appropriate presentation of whatever semantics the underlying application possesses. Presentation is more universal than semantics.

While Deliverance does its best to work with applications as-they-exist, without making particular demands on those applications, it is not perfect. Conflicting CSS can be a serious problem. Some applications don’t have very good structure to work with. You can’t generate any content in Deliverance, you can only manipulate existing content, and often that means finding new ways to generate content, or making sure you have a place to store your content (as in the case of navigation). This is why arguably Deliverance does not remove the need for a platform, but is just its own platform. In so far as this is true, Deliverance tries to be a better platform, where "better" is "more universal" rather than "more powerful". Most templating systems are more powerful than Deliverance transformations. It can be useful to have access to the underlying objects used to procude the markup. But Deliverance doesn’t give you these things, because it only implements things that can be applied to any source of content. Static files are entirely workable in Deliverance, just as any application written in Python, PHP, or even an application hosted on an entirely separate service is usable through Deliverance.

The Missing Parts

As mentioned before, two important benefits of a platform are missing from Deliverance. I’ll try to describe what I believe are the essential aspects. I hope at some time that Deliverance or some complementary application will be able to satisfy these needs. Also, I suggest some lines of development that might be easier than others.

Indexing The Entire Site

Typically each application has a notion of what all the interesting pages in that application are. Most applications have a set of uninteresting pages, or transient pages. A search result is transient, as an example. An application also knows when new pages appear, and when other pages disappear. A site-wide index of these pages would allow things like site maps, cross-application search, and cross-application reporting to be done.

An interesting exception to the knowledge an application has of itself: search results are generally boring. But a search result based on a category might still be interesting. The difference between a "search" and a "report" is largely in the eye of the beholder. An important feature is that the application shouldn’t be the sole entity allowed to mark interesting pages. Manually-managed lists of resources that may point to specific applications can allow people to usefully and easily tweak the site. Ideally even fully external resources could be included, such as a resource on an entirely different site.

To do indexing you need both events (to signal the creation, update, or deletion of an entity/page), and a list of entities (so the index can be completely regenerated). A simple way of giving a list of entities would be the Google Site Map XML resource. Signaling events is much more complex, so I won’t go into it in any greater depth here, but we’re working on a product called Cabochon to handle events.

One thing that indexing can provide is a way to use microformats. Right now microformats are interesting, but for most sites they are largely useless. You can mark up your content, but no one will do anything interesting with that markup. If you could easily code up an indexer that could keep up-to-date on all the content on your site, you could produce interesting results like cross-application mapping.

Shared Authentication And User Accounts

Authentication is one of the most common and annoying integration tasks when crossing platform boundaries. Systems like Open ID offer the ability to unify cross-site authentication, but they don’t actually solve the problem of a single site with multiple applications.

There is a basic protocol in HTTP for authentication, one that is workable for a system like Deliverance, and there are already several existing products (like repoze.who) that work this way. It works like this:

The logged-in username is sent in some header, e.g., X-Remote-User. Some kind of signing is necessary to really trust this header (Deliverance could filter out that header in incoming requests, but if you removed Deliverance from the stack you’d have a security hole).
If the user isn’t logged in, and the application wants them to log in, the application response with a 401 Unauthorized response. It is supposed to set the WWW-Authenticate header, probably to some value indicating that the intermediary should determine the authentication type. In some cases a kind of HTTP authentication is required (typically Basic or Digest) because cookie-based logins are too stateful (e.g., in APIs, or for WebDAV access).
The intermediary catches the 401 and initiates the login process. This might mean a redirect to a login page, and setting a cookie on successful login. The login page and setting the cookie could potentially be done by an application outside of the intermediary; the intermediary only has to do the appropriate redirects and setting of headers.
In the case when a user is logged in but isn’t permitted, the application simply sends a 403 Forbidden response. The intermediary shouldn’t actually do anything in this case (though maybe it could usefully add a logout link to that message). I only mention this because some systems use 401 for Forbidden, which causes no end of problems.

While some applications allow for this kind of authentication scheme, many do not. However, the scheme is general enough that I think it is justifiable that applications could be patched to work like this.

This handles shared authentication, but the only information handed around is a username. Information about the user — the real name, email, homepage, permission roles, etc — are not shared in this model.

You could add something like an internal location to the username. E.g.: X-Remote-User: bob; info_url=http://mysite.com/users/bob.xml. It would be the application’s responsibility to make a subrequest to fetch that information. This can be somewhat inefficient, though with appropriate caching perhaps it would be fine. But many applications want very much to have a complete record of all users. Changing this is likely to be much harder than changing the authentication scheme. A more feasible system might be something on the order of what is described in Indexing the Entire Site: provide a complete listing of the site as well as events when users are created, updated, or deleted, and allow applications to maintain their own private but synced databases of users.

A common permission system is another level of integration. One way of handling this would be if applications had a published set of actions that could be performed, and the person integrating the application could map actions to roles/groups on the system.

Cross-cutting Functionality

This item requires a bit of explanation. This is functionality that cuts across multiple parts of the site. An example might be comments, where you want a commenting system to be applicable to a variety of entities (though probably not all entities). Or you might want page-update notification, or to provide a feed of changes to the entity.

You might also want to include some request logger like Google Analytics to all pages, but this is already handled well by Deliverance theming. Deliverance’s aggregation handles universal content well, but it doesn’t handle content (or subrequests) that should only be present in a portion of pages.

One possible way to address this is transclusion, where a page can specifically request some other resource to be included in the page. A simple subrequest could accomplish this, but many applications make it relatively easy to include some extra markup (e.g., by editing their templates) but not so easy to do something like a subrequest. We’ve written a product Transcluder to use an HTML format to indicate transclusion.

It’s also possible using Deliverance that you could implement this functionality without any application modification, though it means added configuration — an application written to be inserted into a page via Deliverance, and a Deliverance rule that plugs everything together (but if written incorrectly would have to be debugged).

Other Conventions

In addition to this, other platform-like conventions would make the life of the integrator much easier.

Template Customization

While Deliverance handles the look-and-feel of a page, it leaves the inner chunk of content to the application. If you want to tweak something small you will still need to customize the template of the application.

It would be wonderful if applications could report on what files were used in the construction of a request, and used a common search path so you could easily override those files.

Backups and Other Maintenance

Process management can be handled by something like Supervisor, and maybe in the future Deliverance will even embed Supervisor.

But even then, regular backups of the system are important. Typically each application has its own way of producing a backup. Conventions for producing backups would be ideal. Additional conventions for restoring backups would be even better.

Many systems also require periodic maintenance — compacting databases, checking for any integrity problems, etc. Some unified cron-like system might be handy, though it’s also workable for applications to handle this internally in whatever ad hoc way seems appropriate.

Common Error Reporting

With a system where one of many components can fail, it’s important to keep track of these problems. If errors just end up in one of 10 log files, it’s unlikely anyone is closely tracking them.

One product we’re working on to help with this is ErrorEater, which works along with Supervisor. Applications have to be modified to emit errors in a specific format that Supervisor understands, but this is generally not too difficult.

Farming

Application farming is when one instance of an application can support many "sites". These might be sites with their own domains, or just distinct projects. Examples are Trac, which supports multiple projects in one instance, or WordPress MU which supports many WordPress instances running off a single database and code base.

It would be nice if you could add a simple header to a request, like X-Project-Name: foo and that would be used by all these products to select the site (or sub-site or project or any other organization unit). Then mapping domain names, paths, or other aspects of a request to the project could be handled once and the applications could all consistently consume it.

(Internally for openplans.org we’re using X-OpenPlans-Project and custom patches to several projects to support this, but it’s all ad hoc.)

Footnotes

[*]	This isn’t entirely true, Deliverance internally uses WSGI which is a Python-level abstraction of HTTP calls.

[†]	At different times in the past, in an experimental branch right now, and potentially integrated in the future, Deliverance has been compiled down to XSLT rules. So Deliverance could be seen even as an simple transformation language that compiles down to XSLT.

Techcrunch: Sugar High: Sugar Inc Launches API And IMDB For Fashion

Jason Kincaid — 2008-10-07T01:51:06+00:00

Sugar Inc, the company behind a network of popular women-focused blogs that includes PopSugar, is announcing two major releases tonight that take advantage of its fashion-hungry userbase. The first, dubbed PopSugar’s CelebStyle, is positioning itself as an IMDB for style, analyzing outfits from many of television’s most popular shows. Sugar Inc is also announcing a new ShopStyle API, which gives developers access to the site’s massive database of clothing and accessories which can be used in any variety of applications. Here’s a breakdown of both announcements:

ShopStyle API
ShopStyle, which Sugar aqcuired a year ago, is a search engine for fashion. The site indexes clothing and accessories across thousands of brands and stores, presenting them to users in a uniformly formatted grid of thumbnails, prices, and descriptions.

Sugar had previously made this data available to partner sites like In Style and People, and is now opening it up to developers. CEO Brian Sugar says that apps using the API could easily be deployed across a variety of platforms, including webpages (like SaleHabit, which Sugar developed in a weekend), Facebook, and the iPhone.

In thirty days, the platform will also allow developers to convert clicks from the fashion items they’ve displayed into cash. Sugar says the revenue split will vary by app, and will be determined by the success rate of the app’s lead generation.

Fashion may not be most developers’ cup of tea, but the market for this kind of application is huge - expect to see a variety of fashion-based iPhone and web applications hit the market in the next few months.

PopSugar’s CelebStyle

PopSugar’s CelebStyle will allow users to see exactly what their favorite stars have been wearing on some of the most popular shows on TV. The site is edited by a number of Sugar employees, who will pick some of the most prominent outfits from these shows and offer links to each accessory and piece of clothing. To build and help populate the site, Sugar is leveraging the technology and partnerships behind StarBrand, which it acquired in May. Through some of these partnerships (and PR representatives), Sugar editors will have access to lists detailing exactly what the stars were wearing, so users will be assured that the items presented are authentic.

Visitors to CelebStyle will also be able to craft their own outfits using Sugar’s database of clothing, which can be added as comments beneath any article. For instance, users could put together an outfit closely mirroring a fancy get-up shown in the latest episode of Gossip Girl, but with clothes that were only a fraction of the cost.

Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.

Techcrunch: Fight Spam With A Direct Message To Twitter

Mark Hendrickson — 2008-10-07T01:47:35+00:00

There doesn’t appear to have been an official announcement, but Twitter has begun soliciting spam reports to a “spam” user account via direct messages.

Are you a victim of Twitter spam? Just begin following @spam and send it a direct message with the username of your spammer. As the following email autoresponse to spam reporters instructs, you can send these direct messages from your mobile phone or opt for a public tweet as well:

Howdy!

Thanks for reporting spam- we’re working really hard on getting rid of it! Did you know: you can now easily report spam directly from your Twitter account? Visit:

http://twitter.com/spam

and follow the account. You can then send:

* a direct message to @spam: @moneybagsnow is a spammer!
* a direct message from your phone using d+ username + message: d spam @carmoney, @cashnow is spam!
* a reply to @spam like so: @spam this is a spam account: @bigmoney5

and we’ll take care of it. You can send as many spam user names as will fit in one direct message or @reply as long as they are designated like this: @crystal.

Note: it’s better to send a direct message over an @reply. Direct messaging keeps @replies reporting spam out of your followers’ time lines. Sending direct messages also keeps the spam account’s user name out of all search results. Because the message is private, you prevent them from benefitting from publicity. Thanks again for helping us track down spammers!

Twitter Support Team

So far 213 members have begun following @spam, which oddly has decided to follow 179 members itself. It will have to gain a much larger following to make a dent in Twitter’s spam problem. The Twitter Blacklist, a website that tracked all banned Twitter accounts before its owner lost faith in the service, lists 561 blacklisted users as of July 12th.

Read more about Twitter’s spam efforts here.

[Thanks for the tip, Rafa]

Crunch Network: CrunchBase the free database of technology companies, people, and investors

Techcrunch: Geode: The New Geo-Tagging Project From Mozilla Labs

Henry Work — 2008-10-07T00:32:28+00:00

Tomorrow Mozilla will launch a new geotagging project called Geode into Mozilla Labs that promises to leverage your physical location to enhance your overall browsing experience. More details will be provided in an official post tomorrow, but this is what we know already:

Geode is a Firefox add-on that understands location, enabling enriched, personalized, and localized content.

For example with Geode, a user who is looking for restaurants while they are out of town will be able load up their favorite review site and find suggestions a couple blocks away and plot directions there.

It’s unclear how Firefox actually intends to determine a user’s location, especially since Mozilla doesn’t have a mobile browser that could provide GPS data. How web apps are supposed to leverage the plugin is also yet to be seen.

Geode joins the recently announced Labs projects Ubiquity and Snowl. We’ll post more details as soon as we get them.

Crunch Network: CrunchGear drool over the sexiest new gadgets and hardware.

Techcrunch: Forget The Blackberry “Application Center.” The BerryStore Will Have Better Apps.

Erick Schonfeld — 2008-10-06T23:26:44+00:00

As Research in Motion prepares to open its Blackberry Application Center to answer the iPhone’s App Store, an unaffiliated startup called the BerryStore has already launched a competing app store for Blackberry Apps. What makes it better than the official BlackBerry App Center (besides the name), is that apps in the BerryStore work across both old and new BlackBerries alike (not just the upcoming BlackBerry Storm), and across carriers. The BlackBerry App center, in contrast, is designed to be a carrier-specific store, with different apps for different carriers.

AYou can download the BerryStore as an app itself by visiting www.berrystore.com on your BlackBerry. (The App Center will require users to download apps through their Blackberry browsers, which is not the best experience). Already there are about 40 apps in the store, ranging from Loopt, 3Jam, and TwitterBerry to Obopay, Citysense, and Google Mobile. All of them are currently free, although the company plans on offering paid apps in the future. Developers can get more details about how to submit apps or the BerryStore here.

Below is a list of each app currently in the BerryStore with a short description:

Books & Reference
NeoReader: Turns your Blackberry into a barcode scanner.
Blackberry Wiki: Wikipedia reader.
Beyond411: Yellow pages, maps, and directions.
MobipocketReader: Mobile e-reader.

Business & Finance
Obopay: P2P payments.
Bank of America: Manage your dwindling bank account.
NyTimes DealBook: A bookmark icon to the popular blog.
E-Trade Mobile Pro: Manage your dwindling stock portfolio.

Lifestyle
Google Mobile: Search, Maps, Gmail.
Opera: Opera Mini Web browser.
Zumobi:Mobile widgets.
Google Mail: As in Gmail.
Poynt: Local search.
Maufait InstaFind: Al-in-one 411, flight tracker, movie showtimes, stock quotes, weather, news.
Puretracks:Mobile music store.
Tellme: Voice-enabled GPS info.
reQall: Voice-to-text recorder, to-do list, and idea manager.
Nobex Radio Companion: Shows you the name of the songs playing on the radio.

News & Weather
Viigo: News, sports, entertainment, weather, stock and traffic alerts.
New York Times: Bookmark icon.
ABC News: Bookmark icon.
The Washington Post: Bookmark icon.
CNBC Mobile: Bookmark icon.
USA Today: Bookmark icon.
Slate: Bookmark icon.
PinStack.com: Forums

Social Networking
TwitterBerry: Mobile Twitter client that avoids SMS charges.
3jam: Group text messages.
eBuddy: Instant messaging app
Dexrex: Archives your text messages.
Pinger: Voice IM.

Sports
Sports Illustrated: Bookmark icon.
ESPN Mobile: Bookmark icon.

Travel & Navigation
Google Maps: What it sounds like.
GPSed: Map your GPS tracks, save them, and share them. Also geotags your photos.
Citysense: Live hotspot tracking.
WorldMate Live: A personal digital assistant for travelers

Utilities
Box.net: Access and share files on your BB.
AutoLock: Locks the keyboard.
MidpSSH: Connect to remote servers.
MiniMoni: Monitor IP traffic.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

The Scobleizer: Scobleizer

Robert Scoble — 2008-10-06T22:58:52+00:00

Are you depressed about the market? Well, just think of all that we have that costs nothing!

Here, Mitch Joel put together a whole conference for you. Costs nothing. Just grab a glass of wine and watch.

It reminded me of how much I love TED Talks. Watch the video of famous conductor Benjamin Zander, it’ll get your mood lifted.

After that, why don’t you visit the Library of Congress to see how they are saving our old photos and putting them onto Flickr for you to peruse. Don’t think social media matters to something as stodgy as the Library of Congress? Think again!

Our stocks might be worthless, but at least we have good free stuff to watch on the Internet.

That’s the “cup is half full” approach to life.

Bruce Eckel: Photos & Article from Info Magazine (Brazillian)

2008-10-06T22:01:15+00:00

While I was in Brazil, I was interviewed for a Brazillian technical magazine, and the issue was just published.

Bob Ippolito: Mochi Media hiring for Python web developer job in San Francisco

bob — 2008-10-06T21:57:11+00:00

I don't often post about open jobs at Mochi Media on my blog, but it seems that most of the awesome people we have here found out about Mochi Media from me so I might as well ;)

Web development at Mochi Media happens primarily in Python and Pylons with Genshi templates. We're currently using mostly MochiKit for the JavaScript heavy lifting and we talk to all of our backend services (which are either Python or Erlang) via JSON. We're using a bunch of other cool technologies such as memcached, nginx, PostgreSQL, etc. We also often contribute a lot of what we do back to the open source community (e.g. MochiKit, MochiWeb, simplejson, etc.) and would love adding some more team members that were interested in helping out with our open source efforts too! We don't require experience with all of these frameworks and tools, we just need smart people that have a really solid understanding of web development in Python.

Mochi Media runs MochiAds, a monetization platform for Flash games, and MochiBot, an analytics service for Flash content... so we've got tons and tons of data to work with and very interesting scale problems to address. Our primary service, MochiAds, is a monetization platform for Flash games... so part of the responsibilities of being a Mochi employee is to approve all the latest in Flash games for paid advertisements from our network ;) There's even an internal competition to see who approves the most games each week! You can check some of them out at http://www.mochiads.com/games/

Mochi Media was founded in 2005 by Jameson Hsu and myself, and we're backed by Accel Partners and Shasta Ventures. We've got a great team and are always looking to add the best people we can find. If you kick ass at Python and love building cool apps for the web, talk to us!

This position is full-time and on-site in San Francisco, CA. We're very easily accessible by BART and Caltrain in SOMA, at 2nd st. and Mission st. More evidence that Mochi Media is a cool place to work can be found on the mochimedia Flickr photostream: http://flickr.com/photos/mochimedia

More info and application instructions are here: http://www.mochimedia.com/about/careers/#webengineer

Techcrunch: What Does the Slow-Down Mean for Gadget Lovers?

John Biggs — 2008-10-06T21:52:18+00:00

With news of doom and gloom coming at us from every side I would like to take a moment to look at the market as it stands and assess what a slow-down/recession will mean for us, the gadget obsessed. To recap, we are in this mess because of what amounts to a credit freeze. Companies that once could request "cash" to cover overnight expenses - we're talking millions and billions of dollars worth of cash - are now told they can only withdraw a significant percentage less than they were previously allowed to access. This, in turn, reduces capital expenditures and slows hiring - if you can't get a quick loan you can't cover a new factory or a make a bigger payroll. This, in turn, slows large purchases and raises unemployment which, in turn, makes us all freak out.

Techcrunch: Apple Met Goal of 10 million iPhones Sold in 2008

John Biggs — 2008-10-06T21:51:01+00:00

By using some fairly interesting IMEI collection, the folks at Mac Observer have found that Apple sold 10 million iPhones in 2008, reaching and potentially surpassing Steve's original stated goal. By looking at phones sold over the last few months, Mac Observer's "Apple Finance Board" found that the phone has gone through nine 1 million unit runs. Adding this to the known sales they found the total number was far above analyst expectations.

The Google Blog: New Technology Roundtable series

Karen (noreply@blogger.com) — 2008-10-06T21:47:00+00:00

We've just posted the first three videos in the Google Technology Roundtable Series. Each one is a discussion with senior Google researchers and technologists about one of our most significant achievements. We use a talk show format, where I lead a discussion on the technology.

While the videos are intended for a reasonably technical audience, I think they may be interesting to many as an overview of the key challenges and ideas underlying Google's systems. And of course they offer a glimpse into the people behind Google.

The first one we made is "Large-Scale Search System Infrastructure and Search Quality." I interview Google Fellows Jeff Dean and Amit Singhal on their insights in how search works at Google.

The next title is "Map Reduce," a discussion of this key technology (first, at Google, and now having a great impact across the field) for harnessing parallelism provided by very large-scale clusters computers, while mitigating the component failures that inevitably occur in such big systems. My discussion is with four of our Map Reduce expert engineers: Sanjay Ghemawat and Jeff Dean again, plus Software Engineers Jerry Zhao and Matt Austern who discuss the origin, evolution and future of Map Reduce. By the way, this type of infrastructure underlies the infrastructure concepts in our recent post on "The Intelligent Cloud."

The third video, "Applications of Human Language Technology," is a discussion of our enormous progress in large-scale automated translation of languages and speech recognition. Both of these technology domains are coming of age with capabilities that will truly impact what we expect of computers on a day-to-day basis. I discuss these technologies with human language technology experts Franz Josef Och, an expert in the automated translation of languages, and Mike Cohen, an expert in speech processing.

We hope to produce more of these, so please leave feedback at YouTube (in the comments field for each video), and we will incorporate your ideas into our future efforts.

[Cross-posted on the Google Research Blog.]

Posted by Alfred Spector, VP of Research and Special Initiatives

Techcrunch: Meet Mark Zuckerberg In Munich, Tomorrow

Erick Schonfeld — 2008-10-06T21:44:09+00:00

Attention all TechCrunch readers in Munich: here’s your chance to meet Facebook founder Mark Zuckerberg. (Hmm, I wonder if he’ll be making some sort of announcement). He will be giving a speech at a closed, invite-only event on Tuesday evening, October 7 at Amiando headquarters. Amiando CEO Felix Haas is offering to give admission to one lucky TechCrunch reader. (That’s right, only one). Explains Haas in an e-mail:

Mark will give a speech to the selected guests about Facebook’s plans for Europe and Germany. After the speeches and an extensive Q&A session with Mark Zuckerberg there will be plenty of time to get-together with Mark at dinner & drink. I want to offer admission for one of your readers.

All you have to do to apply is explain why you want to meet Zuckerberg in comments. The person with the best reason gets a ticket. (Put in your real e-mail address so Felix can contact you in case you win).

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

Bob Ippolito: simplejson 2.0.2

bob — 2008-10-06T21:30:26+00:00

simplejson (documentation) is a simple, fast, complete, correct and extensible JSON (RFC 4627) encoder/decoder for Python 2.3+. It is pure Python code with no dependencies, but features an optional C extension for speed-ups.

simplejson 2.0.2 is a minor update:

Fixes MSVC2003 build regression
Fixes Python 2.4 compatibility in _speedups.c

Techcrunch: SimplyBox: Drag-And-Drop Clips Of The Web For Research, Sharing

Jason Kincaid — 2008-10-06T21:25:48+00:00

Web bookmarks are badly in need of an overhaul. Sure, they’re fine for jumping to the dozen or so sites you visit on a daily basis, but they quickly become unwieldy whenever they’re used for research purposes. Folders of bookmarked job listings or apartment openings on Craigslists become practically useless as you frequently need to revisit them to remind yourself why you bookmarked them in the first place.

Tonight sees the launch of SimplyBox, a San Francisco-based startup that allows users to clip portions of websites in a visual way, overcoming some of the shortcomings of traditional bookmarks. SimplyBox is currently available as plugins for Internet Explorer and Firefox, with a Safari version on the way. The site will go live at 6 PM PST tonight.

To use SimplyBox, you hit the “Box and Save” button in the browser toolbar, which turns your mouse arrow into a targeting cursor and displays a series of “boxes” at the bottom of the screen. After highlighting the portion of the page you’d like to save for later, you drag it into the box you’d like to store it in. The whole process is very intuitive and only takes a few seconds.

Clicking on the toolbar’s SimplyBox logo brings you to your profile page, where you can access and sort through each of your boxes. Clippings appear as large images that clearly show their contents, and can be displayed in a desktop view (where you can drag them around), a list view, or a grid. Users can also add comments to each of the snippets.

In practice, the plugin seems to work well, especially when conducting research across a number of sites. For example, house hunters could easily use the service to keep track of the homes they’re interested in, clipping photos and summaries for future reference. The site also allows users to share their boxes with friends, so they collaborate on a single project. Finally, the toolbar includes a “box and send” function that sends snippets in Emails as image files.

SimplyBox’s biggest flaw is that its snippets are simple images - they don’t retain any of the functionality of their original source. If you were to snap a clip of a YouTube video, it would only appear as an image, and you’d have to visit the linked page to see the video itself. That said, it’s still a big improvement over simple bookmarks.

There are a number of other “clipping” tools available, including Sazell, Diigo, and to some extent, Safari’s Webclip.

Crunch Network: CrunchBase the free database of technology companies, people, and investors

Techcrunch: GameFly Goes Mobile

Greg Kumparak — 2008-10-06T21:14:59+00:00

GameFly, which is perhaps most easily explained as the NetFlix of the gaming world, today launched a new version of their site specifically optimized for mobile handsets.

Through the magic of user agent detection, getting to the mobile site is a one step process: just type the standard GameFly.com address into the browser of any mobile handset. Once there, users can search through the GameFly library, order games, switch up their "GameQ" queue, read reviews, and manage their account.

IronPython URLs: Compiling the DLR for Silverlight and IronPython with Silverlight RC0

Fuzzyman (noreply@blogger.com) — 2008-10-06T21:06:08+00:00

Silverlight Release Candidate 0 is out, this means that the old versions of IronPython and the Dynamic Language Runtime don't work with the new version.

Silverlight includes a cut down version of the .NET framework called the Core-CLR. IronPython and DLR have to be compiled against the Core-CLR assemblies for the version of Silverlight you are targeting.

But IronPython is Open Source, so you can just compile it against the new Silverlight assemblies right? Jimmy Schementi explains:

Compiling the DLR, IronRuby, and IronPython for ANY version of Silverlight

"Of course! Everything should just work, since there were no major breaking changes in Silverlight that affect the DLR between Beta2 and RC0. So, you hacked up the csproj files to point at mscorlib.dll, system.dll, etc in the new Silverlight install directory (C:\Program Files\Microsoft Silverlight\2.0.30923.0), compile, and it builds fine. Then you try to run an app ..."

Next you need to make a simple change to your dynamic application to tell it about the new version of the runtime. Jimmy covers this in his blog entry as well. Of course rather than compiling IronPython yourself, it is easier to wait for Jimmy to release a new version of the SDK.

Dynamic Languages in Silverlight 2 RC0

Jimmy also explains the changes in the DLR hosting API if you are hosting IronPython or IronRuby in your Silverlight application.

Techcrunch: Granting Credit On The Fly. Is Bill Me Later Part Of the Problem Or Part Of The Solution?

Erick Schonfeld — 2008-10-06T20:56:58+00:00

On the same day the public markets are tanking because of the spreading credit crisis, we see one of the biggest M&A exits of the year with eBay acquiring Bill Me Later for $945 million ($820 million in cash, plus an extra $125 million in options). The only other tech exits of this size in 2008 were Sun buying MYSQL for $1 billion (which involved less cash and more options), AOL buying Bebo for $850 million. This is for a company that lets consumers defer payment when they buy things online. Remember, loose credit is part of the reason we are in the current economic mess.

So is Bill Me Later part of the problem or part of the solution? I put that question to Michael Kwatinetz, the former Wall Street tech analyst who is now a partner at Azure Capital, the biggest shareholder in Bill Me Later. He explained to me how Bill Me Later works, and how it actually has more stringent credit controls than most credit cards:

The problem is people who can’t afford to pay for things are financing things. If you have the proper controls, you don’t allow that to happen.

We don’t grant credit limits. We grant credit on a transaction basis. If you are somebody who is not paying us, or running up your bills in other places, we don’t grant credit.

Traditional credit cards, in contrast, let you run up your bill up to a pre-determined credit limit. With each transaction, BillMeLater check your credit score, credit outsanding, status with credit agencies, and a few other criteria. And it either approves your credit or it doesn’t for each purchase in less than three seconds. Kwatinetz says that the company tightened its lending policies about a year ago, and claims that the nonpayment rate is “probably the lowest of anyone on the Web.”

The value of the company lies in its algorithms that determine credit risk and in its ability to use the Internet as a platform for granting credit. All of the data that its algorithms consider is pulled in via Web APIs from credit agencies and other data providers.

For eBay, Bill Me Later will help expand its Paypal franchise to consumer credit. Bill Me Later is already used all over the Web by more than four million customers at places like Amazon, the Apple Store, JetBlue, and Walmart,com. This year the company is expected to finance $1 billion worth of online purchases and bring in $125 million in revenues. Revenues are estimated to be $150 million in 2009 (a 20 percent growth rate), but the acquisition will also be dilutive to eBay’s earnings, meaning the business is not yet profitable on a net income basis.

Still, you can see the appeal of an all-online credit service. Traditional credit card companies pay about $150 for each customer who signs up for a new Visa or Mastercard, and it usually takes a coupel weeks to process each account through the mail. Bill Me Later acquires customers pretty much instantly at the time of purchase for less than 10 percent of what it costs the credit card companies.

Announcing a large acquisition like this that is essentially a foray into the troubled credit markets may seem counterintuitive, but after a string of deals that didn’t make much sense for eBay (Skype, StumbleUpon), this could end up being one of its smartest non-core acquisitions since Paypal.

CrunchBase Information

Bill Me Later

Information provided by CrunchBase

Crunch Network: CrunchBoard because it’s time for you to find a new Job2.0

The Scobleizer: Scobleizer

Robert Scoble — 2008-10-06T18:06:53+00:00

The DOW is down about 600 about 700 about 370 right now. Yahoo Finance has the bad news. UPDATE: what a bumpy ride. At one point it was down 748 on my screen and ended up down “only” (Yahoo Finance’s headline) 370.

Here’s the deal: many of us are afraid. That fear is changing our behavior (I’m hoarding, aren’t you?). That causes the market to go down.

How does the market stop going down? The death spiral will have to hit the bottom of our fears.

Where do you see the market hitting bottom? I think we’ll test 8,000 by end of the year (some people say probably by the end of the week if it keeps going down line this).

As they say, we have only to fear fear itself. Hope you’re doing OK. This is looking like one heck of a nasty storm on the horizon, isn’t it? 2009 is looking like a pretty tough year.

Anyone have any positive news to get us off of our fears?

Techcrunch: SearchMe Calls Out Yahoo On BOSS Screenshot

Michael Arrington — 2008-10-06T17:41:41+00:00

SearchMe, a new Sequoia-backed search engine that launched in March, isn’t happy with some of the screen shots that have been seen around the Internet promoting Yahoo’s new BOSS (“Build Your Own Search Service”) product.

In particular, this shot, which was included in a CNET article, shows what appears to be SearchMe’s search interface (itself borrowed from Apple’s iTunes album browser) with the logo cut off (compare it to this). CNET describes the screen shot as “One idea Yahoo showed for BOSS: show miniature versions of the Web pages returned by search results.”

To be fair to Yahoo, the screen shot was included in a press briefing document (the slide, which I also received, is shown below) and Yahoo was using it only as an example to show how the service worked. Yahoo has also said that they reached out to SearchMe to discuss a partnership, as they’ve done with scores of other companies. In the briefing, Yahoo didn’t specifically call attention to the slide, and it was made clear to journalists (or at least me) that the product wasn’t made for front end design, just as a search web service.

Yahoo probably should have left the SearchMe logo on the screen shot, but given how many other things there are to criticize Yahoo for these days, I’m inclined to give it a pass.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

Techcrunch: RIM to Launch BlackBerry “Application Center”

John Biggs — 2008-10-06T16:48:51+00:00

You're looking at the BlackBerry Application Center, RIM's answer to Apple's App Store and Google's Android Market. Same basic concept as the other two—you browse and buy applications that enhance your BlackBerry experience. The Application Center—we're running out of variants of the term “application store!”—is set to debut with the BlackBerry Storm software version 4.7. All application data will be stored at the carriers' locale; RIM is totally out of the loop as far as that goes. It's supposed to differentiate the Application Center from the App Store in that regard—carriers can put the applications they want on their own little store.

Voidspace Del.icio.us Links: Smashing Magazine - Fonts and Icons

voidspace — 2008-10-06T16:48:13+00:00

Neil Gaiman: Chapter Six in San Francisco yesterday...

Neil (noreply@blogger.com) — 2008-10-06T16:25:00+00:00

posted by Neil

Last night's reading is up at http://www.mousecircus.com/videotour.aspx?VideoID=6 .

It's Chapter Six, and was really fun to read.

Labels: free book but this time in installments and filmed

Techcrunch: AOL-Yahoo Merger Details Emerge; Deal Could Happen This Month

Michael Arrington — 2008-10-06T16:16:18+00:00

Yahoo is continuing its marathon merger discussions with AOL, sources close to the negotiations have whispered to us, and a deal could happen as early as this month. Is this just a rehash of the reported discussions in February and then again in April?

Yes and no. It’s clear that AOL’s parent company, Time Warner, wants this deal more than ever. What isn’t clear is whether AOL’s assets will fix any of Yahoo’s problems.

The deal structure that is currently being discussed is Yahoo’s acquisition of AOL (content, services and advertising), minus their subscription dial up business. That plus a couple of billion dollars in cash from Time Warner gets them approximately a third of the combined entity. Time Warner’s AOL headache is gone, and they have a stake in the world’s most valuable chess piece in the Google/Microsoft search and advertising war.

Factors favoring a deal: the companies believe Yahoo’s advertising platform would monetize AOL assets far beyond what they’re generating today (a little over $2.4 billion annually). And those against: combined dominance in mail (they’d have 48% of all worldwide email accounts according to Comscore, with Microsoft #2 at 42%) and instant messaging (39% worldwide combined market share, compared to 55% for Microsoft). In reality, though, email and instant messaging market share are only a problem if Microsoft then comes in and buys the combined entity.

Yahoo gets to make a case to stockholders that they dominate the online portal/services/content world, and who cares if they outsource search advertising to Google. Our position is that they can’t succeed in the long run without strong and competitive search advertising, although it may take the Department of Justice to get that message through to Yahoo’s executive team. Even after these entities combine, if they do, Yahoo still has a major long term competitive problem on its hands.

Crunch Network: MobileCrunch Mobile Gadgets and Applications, Delivered Daily.

A Softer World: A Softer World: 361

2008-10-06T15:43:45+00:00

back next

Techcrunch: Tech Stocks Take It On The Chin As Market Continues To Freefall

Erick Schonfeld — 2008-10-06T15:01:08+00:00

It’s a blood bath out there this morning. The S&P 500 is at a four-year low as the credit crisis keeps getting worse, despite the passage of the government’s $700 billion bailout plan. The market is taking tech stocks down with it. Google is down 4 percent to $368, its lowest point since 2006. Apple is down 6 percent to $91. Microsoft is down nearly 5 percent to $25. Amazon, Yahoo, eBay—all down.

Fears of a credit freeze are growing as the contagion spread to banks in Europe. The Fed is already flooding the market with more cash through new powers it was granted in the bailout package. All of this makes you wonder if A) the U.S. government acted fast enough and B) whether the bailout package is going to end up doing any good.

As far as tech stocks are concerned, already as I write this, there seems to be somewhat of a rally going on in some of these stocks (particularly Google) from the lows where they opened. But if the economy falters, tech stocks won’t be a safe haven for investors, even if they are cash-rich and not as exposed to the credit debacle as companies in other sectors. The markets always tend to overreact to systemic risk because nobody knows how far the problems are going to spread. What we are seeing is panic in the face of the unknown. It reminds me of the market panic after 9/11. Investors whop loaded up on tech stocks then ended up making a lot of money.

Does this signal a buying opportunity, or are investors better off running for the hills? Who is buying (or selling) what out there? Tell us in comments.

Crunch Network: CrunchBoard because it’s time for you to find a new Job2.0