A review of the latest patent applications made by Apple, there seems to be a lot of activity going on behind the scenes which have evaded the media's attention and is proving that Apple is far more ahead of the game in their AV development than was previously known.
All photos courtesy of Appleinsider.com
Based on the above graphic, we see the patent for a "Gesture-based control of autonomous vehicles" application applied for on September 19, 2017. This function shows how a system for decision making would be controlled by a passenger's swipe of an arm. The AI would ask the passenger which direction they would prefer to take when two options are available. The solution would be a hand signal made by the passenger. The hand swipe is not just restricted to directional decisions; it is a generic action used for many decisions, such as where to park the car, changing the speed of travel and stopping along the way.
Another feature of decision making would be the additional ways a passenger can interact with the AI; these interactions will include voice commands, eye gaze directions as well as hand prompting.
The hand prompting function will be defined within a designated "interaction zone." This is would be used by the passenger for specific hand movements and would include selecting options by using a finger to making decisions by swiping a hand in the appropriate area.
The use of body movement is an integral part of another application that was made on September 19th, 2017 which was for Apple's "Traffic direction gesture recognition" application. This function differs in one way; it is an external gesture interaction, where a person outside the car can help the AI make a decision using hand gestures. The specific nature is related to police and other emergency personnel that would require the car to move according to their commands. The system would need to be able to differentiate between hand movements for a specific source to general pedestrian and other cars in the general vicinity.
The patent application gives details how the external sensors will detect the presence of an official person outside, or even a passenger that is outside the car, and follow the instructions being given. These commands would then be transmitted by the AV to other AV's close by, allowing them to make decisions based on the incoming data.
Apple also filed on September 21st, 2017 their "Cognitive Load Routing Metric for Vehicle Guidance," patent application. This patent shows how the AI will relate to road conditions to make a route decision. These conditions would include such variables as the number of lanes on a roadway, the existence of traffic signals, the narrowness of a road, and street lights. This information would be added to the real-time vehicular and pedestrian traffic data as well as with other details such as expected weather conditions for each route.
Apple also applied for a "Vehicle Control System" patent on September 18th, 2017. This is their basic control system that uses all the external sensors (LiDAR's and more) to determine the cars position and the position of all objects within the vicinity of the vehicle. This is the basic function of the AV, and this is what is currently being looked at under scrutiny of the Arizona police in the recent Uber accident.
The patent comes with the relevant decision-making flowcharts that show how the self-driving system should work for changing lanes, starting from the intent, to the data received that enables the system to decide how and when it can change lanes based on the incoming data from its LiDAR rack. The decision-making algorithm includes a matrix of probabilities and possibilities that other cars might act, such as overtaking, braking and other sudden actions, which are all factored into how much space the AV needs to secure a safe lane change.
The Apple LiDAR rack
We know that Apple regularly applies for patents, and patents are a funny thing, sometimes they are just made to either mislead competition or to stop someone else from using a specific technological route that could compete with the main patent that is similar in construct and design.
Apple is the second largest AV testing company, after GM is ahead of Uber, Waymo, and others in their road testing numbers. It seems that the Apple Titan project has succeeded until now to remain under the radar of the media, but due to the recent accident in Arizona, where an Uber AV hit a pedestrian at night and killed her, is now focusing the public eye on the integrity of AV AI systems.