News

  • Home > News

        Ford's Windows-based auto tech responds to 10,000 voice commands

        Jonathan Angel | Date: Jul 16, 2010 | Comments: 1



        Ford's Windows CE-based Sync automotive technology will soon be able to understand hundreds more voice commands than before, Nuance Communications says. It's said an enhanced speech engine will let Sync listen for and respond to more voice commands directly, recognize different words that mean the same thing, and integrate a "vast number" of point-of-interest (POI) names and business types into its navigation system.


        Ford's Sync includes Nuance's USA (unsupervised speaker adaptation) technology, according to the Burlington, Mass.-based supplier of voice recognition software. It's said USA learns the voice of a driver within the first three voice commands, quickly creating a user profile and adapting to tone, inflection and even dialect. USA then continues to learn during that same trip, even picking out another user and creating a second profile if the voice is markedly different, Nuance adds.

        Due to present-day limitations in voice recognition technology -- as well as the problems that could ensue if Sync wound up inadvertently eavesdropping on conversations -- the system isn't designed to pick up all speech that occurs within a vehicle. Rather, it responds only to pre-defined phrases.


        Ford's 2011 Flex, offering the MyFord Touch version of Sync

        However, Nuance says, the enhanced "MyFord Touch" version of Sync that will debut this year on the 2011 Ford Edge (above) will recognize hundreds more commands than before, for a total of more than 10,000. (For background on MyFord Touch, see later in this story.)

        Ford Lead Engineer Brigitte Richardson (featured in the video below) is quoted as saying, "As weve gained processing power and learned more about how drivers use the system, weve been able to refine the interface. Customers can do more and say more from the top-level menu, helping them accomplish their tasks more quickly and efficiently."


        A demonstration of enhanced voice recognition in MyFord Touch
        Source: Ford
        (click to play)

        Examples cited by Nuance of voice recognition improvements in the MyFord Touch version of Sync include:
        • More direct, first-level commands
          • Call John Smith dials the phone number associated with John in a connected phones phonebook directly the user isnt required to say Phone first
          • Direct commands related to destinations, like Find a shoe store or Find a hotel, place users in the navigation system menu where they will be walked through the POI search process
          • The command, Add a phone, will enter the phone pairing menu and walk users through the connection process users dont have to enter a phone submenu to initiate the pairing process
        • Quicker, easier entry and search
          • Navigation entries can be spoken as a single one-shot command; for example, One American Road, Dearborn, instead of requiring individual city, street and building number entries
          • Brand names are recognized by the navigation POI menu, allowing drivers to look for chain restaurants, shoe stores, department stores and more, as well as regional and local favorites
          • Direct tuning of radio stations by simply saying AM 1270 or FM 101.1, or using SIRIUS station names or numbers such as 21 or Alt-Nation
        • Use of aliases
          • Within the climate menu, users can voice-request the same function using several different phrases, such as Warmer, Increase temperature or Temperature up helping reduce the need for drivers to learn specific commands
          • When requesting a specific song from an MP3 player, users can now say Play song [title] in addition to saying Play track [title]
        • Personalized access
          • If an occupants USB-connected device, such as an MP3 player, has been named, users can simply say the device name, such as John Smiths iPod, rather than the less personal USB command
        Other recent enhancements to Sync

        With the aid of embedded software supplier Bsquare, among others, Ford has been steadily refining its Windows CE-based Sync technology. Last week, for example, the automaker responded to growing concerns about texting by saying it would provide the system with a "Do Not Disturb" button.

        According to Ford, MyFord Touch (and the essentially similar MyLincoln Touch) will adopt the Bluetooth Special Interest Group's Message Access Protocol (MAP), allowing the system to read incoming text messages aloud to drivers. Since few phones yet support the MAP profile, MyFord Touch also offers a "Do Not Disturb Mode," which block incoming phone calls or text messages from a paired, Bluetooth-enabled mobile phone, saving them on the device for later listening or viewing.

        Even in this mode, drivers will still be able to make voice-activated outgoing phone calls, if they so decide, and the Sync 911 Assist feature can still call emergency services automatically if an airbag is deployed, the company added last week.

        Ford also announced it is "taking the proactive step of locking out or limiting content and capabilities of MyFord Touch that are not related to the task of driving when the vehicle is in motion. Locked-out functionality was said to include:

        • Album cover art and phonebook photo browsing
        • Destination entry by touchscreen keyboard input (voice entry is permissible)
        • Editing photos
        • Editing the screens wallpaper or adding a new one
        • Manually adding individual contacts into the honebook
        • Pairing a Bluetooth-capable phone
        • Playing video
        • Viewing received text messages on screen
        • Web browsing on the vehicle screen
        In April, meanwhile, Ford announced that its Windows CE-based Sync technology would be upgraded later this year to allow in-car "tweeting" using voice commands or steering wheel buttons. The company said its "AppLink" feature would tie the automotive infotainment system to smartphone apps updated using a Sync API (application programming interface), including software for Pandora and Stitcher radio applications, plus Orangatame's OpenBeak app for Twitter.

        More about Sync ...

        Ford first introduced its Windows CE-based Sync automotive technology in 2008-model cars, initially focusing the system on integrating phones and PMPs (portable music players) into a car's entertainment system. As we detail below, the system has subsequently been broadened to include GPS, cellular data, and a variety of other elements.

        According to Ford, Sync first began to ship in fall 2007 and has now been installed in more than two million Ford, Lincoln, and Mercury vehicles. This milestone was reached only ten months after the one-millionth Sync-equipped vehicle, a 2010 Ford Fusion Hybrid (below), was presented to Microsoft CEO Steve Ballmer, Ford says.


        The 2010 Ford Fusion Hybrid

        At January's Consumer Electronics Show (CES), meanwhile, Ford showed off the Sync upgrades that will featured in some 2011-model cars, adding dual 4.2-inch displays and an available eight-inch touchscreen. "MyFord" and "MyFord Touch" will include navigation with turn-by-turn directions, and will allow installation of third-party applications, according to the company.

        Ford said at CES that the MyFord infotainment system would build on previous versions of Sync by adding two 4.2-inch color LCD screens. One screen (shown, below left), in the instrument panel, will be controlled using a cellphone-style five-way controller, and will provide access to the following, the company says:

        • Turning driver assist technologies -- traction control, blind spot information system, cross traffic alert, cruise control, front park assist, rear park assist, and trailer sway control -- on and off
        • Changing vehicle settings such as automatic lamp delay, locking, remote start, remote window open/close, and rain-sensing wipers
        • Monitoring engine speed, fuel economy, distance-to-empty, and miles traveled

        The dual displays offered by Ford's MyFord system

        The other screen (shown above right) will provide a visual representation of turn-by-turn navigation, with turn arrows, street names, and distance-to-turn information. As a result, drivers will no longer need to pay for GPS hardware upgrades or third-party receivers, says Ford.

        Meanwhile, an enhanced version of the system called "MyFord Touch" will add an eight-inch touchscreen display in a vehicle's center stack, plus a hub (pictured below left) below it that offers audio and video inputs, two USB ports, and an SD card reader. Ford says motorists will be able to purchase SD cards containing maps, enhanced POI (point of interest) information, and voice data, making it easy to upgrade to 3D maps (below right) that are regularly updated.


        The media hub and navigation screen offered by Ford's MyFord Touch system

        As well as being used for an enhanced navigation display (above), the eight-inch touchscreen can also display: driver-selected wallpaper (pictured top right); frequently accessed settings such as phone functions, navigation destinations, climate control settings or audio controls (pictured, middle right); or driver-selected "hot buttons" that can be tied to almost any controllable system in the car (pictured, bottom right).

        Display settings are memorized and stored according to a driver's unique key, allowing each user of a car to customize the screens to his or her own preferences, Ford added.

        As mentioned earlier in this story, Ford officials promised that the MyFord Touch screen will also be able to display third-party applications that have been loaded onto the system, such as social networking (Twitter) and internet radio (Pandora and Stitcher) clients. Such applications, enabled via a newly available API (application programming interface), will work via a driver's mobile phone, which provides the system with its wide area connectivity.

        As well as partnering with Pandora, Stitcher, and Twitter, Ford said, it has also allied with MapQuest, so that vehicle owners can use their personal computers to plan a journey, then send the data to their cars using a simple "send to Sync" command. According to the company, the information is received in-car by a driver's mobile phone using voice technology; therefore, a user does not have to have a data plan or even own a smartphone.

        ... and more about Microsoft's auto technology

        Microsoft's Windows CE for Automotive was first released in 1998, since when it has evolved into a variety of product offerings. For example, in 2003 it was called Windows Automotive (2003), and by 2006 it was known as Windows Mobile for Automotive.

        Unlike with many Microsoft products, however, naming is secondary, since the core technology is designed to be adapted by automaker customers and offered under their own brands. For example, the Windows CE-based software made its debut in Europe and South America on Fiat vehicles, where it is known as Blue&Me.

        The technology was later released in the U.S. by Ford as Sync, on passenger cars, and Ford Work Solutions, for commercial vehicles. It is also offered in a version for Peterbilt and Kenworth big rigs.

        In March 2009, Microsoft announced Microsoft Auto 4.0, which it said would be made available in the spring to automakers. The new version added a standard interface for broadcast radio tuners, CD playback and ripping support, and support for x86 processors such as the Intel Atom Z530, Z510P, Z530PT, and Z510PT.

        According to Microsoft, the latest release of its software is now Microsoft Auto 4.1, which is based on Windows CE 6.0 R3. Version 4.1 includes the following enhancements, according to the company:

        • Bluetooth 2.1 + EDR
        • Secure Simple Pairing support
        • Calendar download from mobile phone
        • Technology preview of upcoming support for DLNA media device integration
        • Multi-application access to media index
        • iPhone/iPod Touch firmware 3.x support
        • Message Access Profile (MAP) 1.0
        Meanwhile, Microsoft plans a further upgrade that, among other things, will bring Microsoft Auto into line with Redmond's preferred "7" nomenclature. Presumably still using a Windows CE core, Windows Embedded Automotive 7 will launch during the third quarter of this year, according to a slide (below) that appeared on a Microsoft EMEA press website.


        Windows Embedded Automotive 7: set for third-quarter release
        (Click to enlarge)

        Further information

        More information on Microsoft's Auto 4.1 software may be found on the company's website, here.

        More information on Sync may be found on the Ford website, here.

        More information on Nuance's voice recognition products may be found on the company's website, here.


        Related stories: