AUDERTEC SOLUTIONS LLP vs CONTROLLER GENERAL OF PATENTS, DESIGNS AND TRADE MARKS & ANR.

$~(Original)
* IN THE HIGH COURT OF DELHI AT NEW DELHI
+ C.A.(COMM.IPD-PAT) 3/2021
AUDERTEC SOLUTIONS LLP ….. Appellant
Through: Ms. Priya Adlakha, Mr. Bindra
Rana, Ms. Rima Majumdar, Mr. Dhruv Mathur and Mr. Swaraj Singh Raghuwanshi, Advs.

Versus

CONTROLLER GENERAL OF PATENTS,
DESIGNS AND TRADE MARKS & ANR. ….. Respondents
Through: Mr. Harish Vaidyanathan Shankar, CGSC, Mr. Srish Kumar Mishra, Mr. Sagar Mehlawat and Mr. Alexander Mathai Paikaday, Advs.
CORAM:
HON’BLE MR. JUSTICE C. HARI SHANKAR
J U D G M E N T
% 01.03.2024

C.A.(COMM.IPD-PAT) 3/2021

1. The appellant submitted Application No. 202011011938 dated 19 March 2020 for grant of a patent in respect of an invention titled “a method and system for detecting road anomalies” (hereinafter “the subject patent”). The application stands rejected by the Controller of Patents and Designs (“the Controller”) vide order dated 8 January 2021 passed under Section 15 of the Patents Act, 1970. The appellant is in appeal against the said order.

2. The impugned order rejects the application on the ground that the claim in the application suffers from want of inventive step vis-a-vis prior art D-2. Though the First Examination Report (FER) dated 15 June 2020 cites four prior art documents D-1 to D-4 as disclosing all the features of the claims in the subject patent, the notice of personal hearing, issued consequent to the reply filed by the appellant to the FER, restricted the allegation of lack of inventive step to comparison of the subject patent with the prior art D-1 to D-3 and the final impugned order holds the subject patent to be lacking an inventive step only vis-a-vis the prior art D-2.

3. I may note that this position was accepted by both sides and arguments were also advanced before me, on the aspect of obviousness and inventive subject in the subject patent vis-a-vis the prior art D-2.

4. The Court is only required, therefore, to examine whether the subject patent is lacking an inventive step vis-a-vis D-2; in other words, whether the distinguishing features of the subject patent would be obvious to a person skilled in the art from the disclosures contained in D-2 and, therefore, whether the impugned order is correct in rejecting the appellant’s application on the ground of lack of inventive step.

5. For this, one has to appreciate the essential features of the suit patent, which are claimed, by the appellant, to be inventive.

6. The complete specifications of the suit patent read thus:

FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003

COMPLETE SPECIFICATION

(See section 10 and rule 13)

1. TITLE OF THE INVENTION

A METHOD AND SYSTEM FOR DETECTING ROAD ANOMALIES
2. APPLICANT

a) AUDERTEC SOLUTIONS LLP;
b) Indian;
c) SCO 315-316, First Floor, Himalaya Marg, Sector 35B, Chandigarh – 160036, India
3. PREAMBLE TO THE DESCRIPTION

The following specification particularly describes the invention and the manner in which it is to be performed.

FIELD OF THE INVENTION

[001] The present invention relates to transportation management information system. More particularly, it relates to a system and method to maintain the heavily travelled roadways.

BACKGROUND OF THE INVENTION

[002] In the recent years, with exponential increase in traffic on the roads, there is lot of pressure on continuous upkeep and maintenance of the road networks across the cities. Road authorities are losing the continuous battle for upkeep of the road because of frequent weather change, improper drainage, improper or inadequate repair work, improper design of the road, frequent cutting of the roads for laying cables and conduits, heavy traffic or heavy vehicles plying on the roads which are not designed to handle such loads. As a result, road imperfections develop continuously and need to be managed before they become dangerous.

[003] Road anomaly may be defined as misalignment in the road surface due to any cut or crack in the road due to cable laying or any pothole due to improper water drainage or some 20 bump or uneven level due to manhole cover which leads to sudden braking or turning of the vehicle as soon as driver tries to negotiate the anomaly or sometimes if the driver is not able to negotiate then it leads sudden jerk in the vehicle.

[004] In both the above situations either vehicle damage could happen, or accident can occur due to sudden application of brakes or turning of the vehicle. In some cases, skidding of the vehicle because of sudden braking or turning over has led to fatalities. Hence, it becomes utmost important to continuously monitor the road surface condition for anomaly and make sure that the road condition should not become worse to cause accident or unfit for driving. Also, the roads can be maintained better if the anomalies are detected and fixed in the early stages due to less cost of preventive maintenance as compared to rehabilitation or upgrading or reconstruction of the roads.

[005] Further, USA patent US9863928 B1 discloses a road condition detection system for identifying and monitoring road conditions, and for communicating information regarding road conditions to various users. The road condition detection system is provided for capturing data indicative of road conditions and analyzing the captured data to locate and identify various road conditions (e.g., road hazards, such as potholes, or weather conditions, such as ice). In various embodiments, the road condition detection system includes a road condition sensor array configured for being attached to a vehicle and for capturing road condition data. The captured data may be transmitted and assessed by a server configured for identifying potential road hazards or other road conditions based on the road condition data captured by the sensor array. The prior art discloses a system based on laser and vibration sensors. The system however, does not detect the severity of the road condition.

[006] Another USA patent US20180068495A1 discloses a method of detecting and identifying road surface defects is provided. Motion and position information is received from a plurality of vehicles. A profile is retrieved for a particular vehicle from a database of vehicle profiles by using an identifier of the particular vehicle. One or more criteria are identified for detecting a particular type of road surface defect based on the retrieved profile of the particular vehicle. Upon determining that the received motion and position data satisfies the identified criteria, a detection of a road surface defect of the particular type and a location associated with the detected road surface defect based on the received position information is reported. The prior art discloses method of detecting the road conditions using motion sensors and accelerometer. The method disclosed in prior art is cost intensive as multiple vehicle types and their sensor data has to be correlated based on their weight, tyre size, dimensions etc.

[007] Nowadays, various methods and systems are available based on the technologies related to sensors and laser. These methods and systems have limitations due to vibration-based sensors and high costing of laser scanning. Also, the laser sensor is not efficient in wet weather and narrow roads. Due to these limitations there is a need of cost effective method and system for detecting the road anomalies.

OBJECTIVE OF THE INVENTION

[008] The primary objective of the present invention is to provide a method and system to give prior warning to the subscribed driver or user for upcoming road issues.

[009] Another objective of the present invention is to provide a cost-effective method and system for better surveillance of road condition.

[0010] Yet another objective of the present invention is to automate the measurement of road condition.

[0011] Yet another objective of present invention is to provide safe driving assistance to its subscribed users.

[0012] Another objective of the present invention is to provide a proactive system and a method to report road anomalies regularly.

[0013] Yet another objective of the present invention is to provide a method and for periodic tracking of the road conditions which may help in accessing the quality of re-carpeting or patch work done.

SUMMARY OF INVENTION

[0014] The present invention proposes a system for tracking the road anomalies through a dashcam mounted on plurality of vehicles. The method and system maps and identifies the road anomalies by analyzing the video of road conditions and classify the road anomalies by earmarking the road anomalies into potholes, bumps, cracks, etc. and rating them into yellow, amber and red colors based on certain parameters like dimensions of anomalies, type of anomaly, type of road on which anomaly has happened(highway, main city road, colony road, service road, etc.), GPS co-ordinates, date time stamp when the anomaly has been reported, changes if any since first time the anomaly has been reported, area to which the location belongs (sector, area, wards, zones, sub-district, tehsil/taluka, etc.).

BRIEF DESCRIPTION OF THE DRAWINGS

[0015] A complete understanding of the present invention may be obtained by reference to the accompanying drawings, when taken in conjunction with the detailed description thereof and in which:

[0016] Figure 1 illustrates the layout of various components of the system for detecting the road anomalies.

[0017] Figures 2(a) and 2(b) illustrates the functioning of data capturing unit for detecting road anomalies.

[0018] Figures 3(a), 3(b), 3(c) and 3(d) illustrate the process of classification of road anomalies and mapping of road anomalies on to a web mapping service application.

[0019] Figures 4, 5 and 6 illustrates the process of publishing of road anomalies to users.

DETAILED DESCRIPTION OF THE INVENTION

[0020] The following presents a simplified description of the invention in order to provide a basic understanding of some aspects of the invention. This description is not an extensive overview of the present invention. It is not intended to identify the key/critical elements of the invention or to delineate the scope of the invention. Its sole purpose is to present some concept of the invention in a simplified form.

[0021] Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope of the invention. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness

[0022] Features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments and/or in combination with or instead of the features of the other embodiments.

[0023] The terms and words used in the following description and claims are not limited to the bibliographical meanings, but, are merely used to enable a clear and consistent understanding of the invention. Accordingly, it should be apparent to those skilled in the art that the following description of exemplary embodiments of the present invention are provided for illustration purpose only and not for the purpose of limiting the invention.

[0024] It is to be understood that the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise.

[0025] It should be emphasized that the term “comprises/comprising” when used in this specification is taken to specify the presence of stated features, integers, steps or components but does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof. The equations used in the specification are only for computation purpose.

[0026] In accordance with the present invention, Fig. 1 shows system for detecting road anomalies. The various components of the system for detecting road anomalies are, data capturing unit (100), data recording unit (200), data processing unit (300), data mapping unit (400) and data publishing unit (500):

-Data Capturing and Data Recording Unit (100 and 200) may comprise a High Definition dash board camera mounted on top of a vehicle, which is driven at a speed of 20-30 km/hour at a pre-decided route set by the user. The dashboard camera may be mounted on front or back of the vehicle, pointing towards the road surface to capture video of the road surface. As the vehicle moves, the camera continuously records the road conditions through its lenses, covering a 130-degree view. The cameras provide a high definition view of the road surface and simultaneously the GPS data is recorded by the GPS data recording unit (200). Once the survey is done by the camera, the camera footage and GPS log files (.GPX) are downloaded to a central computer to compute the same.

– Data Processing Unit (300) may comprise of the central system to process the raw videos and identify and classify the road anomalies based on training dataset of the road anomalies. This process also helps in classifying the severity of the road anomaly data and notifying the road authorities and subscribed drivers in case of potential issues. This process also gets the input of the GPS data like the deceleration in speed at a particular spot or patch of the road.

– Data Mapping and Publishing Unit (400 and 500) may comprise the geographical mapping of road anomalies, creation of anomalies database and its updation based on road survey conducted, plotting of the road anomalies on any available web mapping service based on the route selected by the drivers along with the name of the roads featuring distress spots and suggesting alternate route with their anomalies information. Further, the report generates a map plotting the exact spots of road distress with color coding based on severity of road anomalies on the web map and create a heat map to show the concentration of road anomalies in the area or zone. This helps both the road authorities to take necessary actions on preventive maintenance and the subscribed drivers take the conscious decision on choosing the appropriate route.

[0027] Fig. 2(a) and 2(b) illustrate the functioning of data capturing unit (100) for detecting the road anomalies. The data capturing unit (100) comprises a dashboard camera (Dashcam), a Global Positioning System (GPS) and a storage device. The Dashcam is a video camera with high definition 1080p, 130 degree viewing angle covering all lanes of the road, Wide Dynamic range (WDR) function which adjusts according to the ambient light, built in G-sensor for image stabilization in case of shock or jerk. The Dash cam may be but not limited to, Akaso dash cam C330. Dashcam records the videos of the roads to analyses the road anomalies.

[0028] In an embodiment, the dashboard camera and GPS may be combined.

[0029] Further, Dashboard camera may pair with a communication device through available network to transfer data to the mobile application. Afterwards, the data from mobile application can be transferred to central server through any network. Alternatively, the data may be physically transferred to central server via storage device such as Secure Digital (SD) card.

[0030] The GPS data recording unit (200) or a GPS logger is a device which is used to capture the GPS co-ordinates and create a stream of it so that once the whole route is recorded the data can be used to trace the vehicle’s path on the map, the date time stamp and the speed at which the vehicle has traversed the route. The GPS logger may comprise of GPS receiver, a processing and storage unit. In an exemplary embodiment, U-blox7 GPS Receiver, a Raspberry PI model 4 B and 16 GB SD card for recording the stream of GPS co-ordinates are used. The data recorded from GPS logger is stored as .GPX file. With every
track being recorded as a .GPX file it is easy to play it in GPX player software which plays the recorded location position along with date time stamp and the speed. The captured .GPX files can be uploaded from Raspberry PI to central server through SSH protocol in a secure manner for processing at the central server. The GPS logger will help create a trace of the route travelled by the vehicle used for road survey. The GPS co-ordinates of potholes and patches are identified and mapped to identify the accurate position of road issues.

[0031] Further, the data capturing unit (100) installed on plurality of vehicles to capture the video and GPS data. Vehicles may be but not limited to, any utility vehicle deployed by municipal committees, personal cars, or commercial vehicles such as trucks, buses etc. but preferably plying during the less traffic time in the morning when shadows are less prevalent.

[0032] Fig. 3(a), 3(b), 3(c) and 3(d) illustrate the process of classification of road anomalies and mapping of road anomalies on to a web mapping service application.

[0033] As shown in Figure 3(a), the central server is a high end desktop machine which process the data captured by data capturing unit (100). In an exemplary embodiment, a desktop machine with Intel i7 processor, Nvidia GPU, atleast 16-32 GB RAM, 512 SSD is used. The central server runs an Artificial Intelligence based algorithm to identify the road anomaly based on the trained dataset of road anomalies. Training data set categorize anomalies in different categories such as potholes, cracks, scrapped road, bumps, misalignments of manhole covers and frames. Further, the server classify the quality of roads as fair, bad or worse and determine the right speed to cross over the anomalies based on recent driver experience. It also determines the alternate travel path for users to avoid travelling over a road having anomalies.

[0034] Further, as shown in figure 3(b) and 3(c), the video is processed through the data processing unit (300) to detect the road anomalies. After processing the video, image frames are recorded having bounding box corresponding to the detected anomalies along with date and time stamp. A bounding box is an imaginary box to demarcate the objects from their surroundings. In digital image processing, the bounding box is merely the coordinates of the rectangular border that fully encloses a digital image when it is placed over a page, a canvas, a screen or other similar bi-dimensional background. Based on the dimensions of the anomalies and the data of deceleration on location of the road anomaly, the road anomaly is classified into fair (yellow), bad (amber) or worse (red). If there are multiple road anomalies in the particular stretch of the road the whole patch may be marked as bad or worse.

[0035] The data mapping unit (400) classifies and labels the road condition by creating a map of road including road anomalies and provide the average deceleration in speed to negotiate the road anomaly. While the driver sees the different routes on the map for reaching the destination the driver may also choose to select the route which has less road anomalies.

[0036] In accordance with the present invention the method for detecting road anomalies is discussed herein:

Step 1: Installation of the dashboard camera and GPS logger in the vehicle;

Step 2: Starting the dashboard camera and GPS logger simultaneously when the vehicle starts surveying the road;

Step 3: Dashboard camera will record the road condition and GPS logger will start capturing the distance covered, time taken and speed at each point and creates GPS log file (.GPX file).

Step 4: The data captured through dashboard camera and GPS logger is stored in separate local storage device.

Step 5: The Data stored in the local storage device is transferred to data processing unit (300).

Step 6: The video is processed through the data processing unit (300) to detect the road anomalies and create a bounding box and capture the image frame along date and time stamp.

Step 7: The data timestamp of image obtained from the (.GPX file) with road anomaly is compared with the corresponding time stamp through GPS logger and the GPS co-ordinates of the road anomalies based on corresponding GPS time stamp are determined.

Step 8: The road anomaly is further geotagged as red or amber or yellow to bring to the attention of the user about the condition and severity of the anomaly on web
mapping application using the GPS coordinates which will show up as pin on map.

Step 9: The said user can click on the pin to see the actual image of the road anomaly with the date time stamp. User will be able to see the severity of the anomaly and average speed for the patch based on drivers travelling at off peak hours.

Step10: There might be a date time stamp based and area wise heat map created to segregate the different areas where there are more anomalies as compared to other areas.

[0037] Step 1 to 5 facilitate the data capturing of required data. Step 6-7 is related to the process of detecting of road anomalies. Step 8 is based on mapping of road anomalies on the web mapping service application. Steps 9-10 are facilitating the publishing of road anomalies to be detected by the users.

[0038] In accordance with the present invention the said process for classification of the road anomaly and determining its severity and accordingly determining the speed of vehicle on the same road is discussed herein:

* The severity of the road anomaly is classified based on correlation of the video of the road labelled through data mapping system, exact geotagging of the location and average decelaration of speed during off peak traffic on the same spot.

* Tagging of road anomalies in different color scheme like red or amber or yellow to bring to the attention of the road authorities about the severity of the anomaly, so that the prioritizing and planning of repair work by authorities can be carried out.

* Historical analysis of the deceleration of the speed through the same location where the anomaly is detected.

* Informing the user in advance about the severity of the road anomaly and the average speed for that patch of road.

[0039] The process of classification of the road anomaly and determining its severity and the amount of deceleration of speed from the average speed on the same road is a continuous and iterative process which will keep on geotagging the new spots or changing the colors and removing the geotags if the road is re-surfaced or patch work done.

[0040] In accordance with the present invention FIG. 4, 5 and 6 illustrate the process of publishing of road anomalies to users.

a) Process for visualization of the road conditions based on specific area includes:

* Collecting and processing the data of road anomalies for a particular area for a period of every 10-15 days to show the variation in data over a period of time.

* Geographically color code the sub areas within area based on number of anomalies and their severity.

b) Process for identifying the route, automatic downloading of the road anomalies data for the route and providing the suggestions if the road quality is really worse consists the following:

* User switches on the web mapping service application and selects the destination and presses start.

* System identifies the current location and the destination location and selects the best route.

* System downloads the road anomalies data for the route from the current database of road anomalies and shows it visually to the subscribed driver on web mapping service application.

* The driver is given warning of the upcoming road anomaly based on its severity and subscribed driver’s speed (only on mobile application). The system also tells about the average speed at which driver should cross the particular patch of road based on past few days’ data for off peak traffic.

c) Process of warning the driver (only on mobile application) in case the upcoming road anomaly is severe or driver is at higher speed as compared to average speed in off peak hour includes:

* Subscribed user’s speed is calculated based on GPS logger installed and route is detected.

* The route currently being undertaken is compared against road anomaly’s database and the average speed for past few days for off peak hour.

* Driver is given audio warning if the driver’s speed is much higher for approaching anomaly.

[0041] The method of publishing the information about upcoming road anomalies may give visual and audio alert to subscribed user in case the driver is at higher speed then prescribed speed for the respective patch.

[0042] In accordance with the present invention the advantages offered by the present invention are:

* Enabling easy survey of roads for locating distress spots, requiring minimal human involvement.

* Leveraging technology to efficiently cover a large area within the monitoring framework within less time.

* Leveraging technology to capture any other incident along the streets like dumping of construction waste, spilling of waste, collection of water, etc.

* Enabling report generation that details out the near-exact location of the potholes and other distress spots on the scanned roads, thus making it convenient for the city authorities to identify the locations as well as assess the distress severity, and accordingly take prompt action.

* The present invention uses a dashcam a specialized camera having wide angle of 130 degree, G sensor for image stabilization for capturing road conditions and subsequent video analysis through AI algorithm for detecting and classifying the road conditions and correlating with the data on deceleration in speed from multitude of vehicles at the same spot on the road irrespective of traffic condition and other adverse conditions for further classifying the road conditions. The present invention does all the processing centrally and focuses on analysis of pothole and road surface anomalies.

[0043] While this invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

We Claim:

1) A method for detecting road anomalies, comprising:

– installation of the dashboard camera and GPS logger in the vehicle;

– starting the dashboard camera and GPS logger simultaneously when the vehicle starts surveying the road;

– recording the road condition through the dash board camera and capturing the GPS co-ordinates through GPS logger and create GPS log file (.GPX file);

– storing the data captured through dashboard camera and GPS logger in separate local storage devices

– transferring the video and GPS data captured by data capturing unit (100) to the data processing unit (the central server)(300);

– processing the video captured from the data capturing unit (100) to identify the road anomaly by matching with trained data set of road anomalies by the server;

– creating a bounding box around the detected road anomaly;

– capturing an image frame along with date time stamp for corresponding bounding box;

– segregating the detected road anomalies and classifying the severity of road anomalies;

– determining the GPS coordinates of road anomaly by matching the date time stamp data of anomaly detected with corresponding date time stamp data received from the GPS log file (.GPX file);

– determining the suitable speed to cross an upcoming road anomaly based on data captured for past few days; and

– mapping of road anomaly on a web mapping service application using GPS coordinates along with information of speed and severity of road anomaly.

2) The method as claimed in claim 1, wherein detected road anomalies can be potholes, bumps or cracks or any other road pavement (surface) misalignment.

3) The method as claimed in claim 1, wherein said method classifies the severity of road anomalies based on different parameters including dimensions of anomalies, type of anomaly, type of road on which anomaly is detected.

4) The method as claimed in claim 1, wherein said method earmark the severity of road anomalies by different colors.

5) The method as claimed in claim 1, wherein the said method notifies user for safe speed through visual and audio alert message.

6) A system for detecting road anomalies, comprising:

-the data capturing unit (100) installed on the vehicle to capture the video and GPS data;

-the data recording unit (200) to store the data acquired by data capturing unit (100);

-the processing unit (300) having a server configured to process the data received from the data capturing unit (100);

-data mapping unit (400) to mark the location of detected anomalies on the web mapping service application; and

-data publishing unit (500) to notify the information regarding suitable speed and severity of road anomaly to the user;

7) The system for detecting road anomalies as claimed in claim 6, wherein the data capturing unit (100) includes at least one dashboard camera, at least one GPS logger and at least one storage device.

8) The system for detecting road anomalies as claimed in claim 6, wherein the GPS logger includes a GPS receiver, a single board computer processing unit, and a power source.

Dated this 19th day of March, 2020
Sd./-
RANA, Vikrant (PA-248)
of S. S. Rana & Co.
Attorney of the Applicant
AUDERTEC SOLUTIONS LLP”

7. One may simplify the features of the subject patent as they emerged from the complete specifications reproduced (supra), thus:

(i) The subject patent is intended to detect road anomalies. The expression “road anomalies” is a compendious term covering any defect in road surface, owing to any cut and crack in the road, pothole, bump or uneven level, which would require a driver driving on the road, to suddenly brake or turn the vehicle or if he does not do so, to a sudden jerk in the vehicle. As this can result in accident, mapping of road anomalies is necessary, both for notifying the concerned authorities, who can rectify the roads as well as to inform subscribed drivers of such anomalies, so that they can negotiate the vehicle appropriately or choose roads, which are more easily negotiable.

8. Five objectives of the claimed inventions are identified, viz.

(a) Providing a method and system to give prior warning to a subscribed driver or user of upcoming road issues, (b) providing a cost effective method and system for better surveillance of road condition, (c) automating measurements of road condition, (d) providing safe driving assistance to subscribed users, (e) providing a proactive system and a method to report road anomalies regularly and (f) providing a method for periodic tracking of road conditions.

9. The method claimed in the subject patent, when adopted, fulfils two purposes. Firstly, it analyzes the video of the road so as to map the conditions of the road. Secondly, it classifies road anomalies by earmarking them on the basis of the type of anomaly and rating the anomaly as yellow, amber or red, based on pre-determined parameters. Thirdly, the complete specifications are accompanied by drawings, designated as figures 1, 2(a), 2(b), 3(a), 3(b), 3(c), 3(d), 4, 5 and 6. These drawings, followed four steps:

(a) Detection of the anomalies by a Data Capturing Unit,
(b) Classification of the anomalies,
(c) Mapping of the anomalies,
(d) Publication of the data to users

I may note, here, that when one peruses Claims 1 to 8 in the suit patent – already reproduced (supra) – it is clear that fourth step noted hereinabove, i.e. publication of the data to users is not part of the claims in the subject patent.

10. The claimed invention is divided in complete specifications, into:
(a) Data Capturing and Data Recording Unit,
(b) Data Processing Unit and
(c) Data Mapping and Publishing Unit.

11. The Data Capturing and Data Recording Unit comprises the following features:

(a) The unit has:
(i) a camera,
(ii) a Global Positioning System (GPS) and
(iii) a storage device.

(b) The camera has
(i) a WDR function which adjusts to ambient light
(ii) a G – sensor for image stabilisation in the case of jerk.
(iii) The camera is a high definition dashboard camera mounted atop the vehicle at the front or the rear. It points towards the surface of the road. It has a 130 degree viewing lens. It can be mounted at a multitude of the vehicles.

12. The vehicle is to be driven at a speed of 20 to 30 kilometre per hour at a pre-decided route set by the user. As the vehicle moves, the camera records a video of the entire road surface, including any anomaly in the road covering of 130 degree view.

13. The GPS data of the road is simultaneously recorded on the GPS recording unit.

14. The GPS Data Recording Unit – (hereinafter the GPS DRU) – alternatively called the GPS logger – consists of a GPS receiver and a processing/storing unit.

15. The GPS DRU captures the GPS coordinates of the road and creates a stream so that it is possible to trace the path of the vehicle, the date time stamp and the speed of the vehicle.

16. The data is stored as a .GPX file, which can easily be played using GPX player software which, when played, would identify the position of the vehicle, date and time stamp and the speed.

17. Thus, the GPS DRU identifies anomalies and maps them so that accurate locations are identified.

18. The camera footage and the GPS log files (in .GPX format) are downloaded to a central computer.

19. Alternatively, the camera may be paired with a mobile through a mobile application, and the mobile can send the data to the central server. Again, alternatively, the video recording recorded by the camera can be physically transferred to the central server through a Secure Digital (SD) Card.

20. The Data Process Unit consists of a central server, which is a desktop machine that processes the data captured by the camera and the GPS DRU and mapped by the GPS DRU.

21. The central server identifies road anomalies based on a dataset. The trained dataset classifies the anomalies into various categories such as potholes, cracks, scrapped road, bumps, misalignments of manhole covers, and the like.

22. The central server thereafter classifies the road, based on the nature of anomalies contained therein as fair (yellow), bad (amber) or worse (red) and, thereby, identifies the quality of road by colour coding.

23. Based on this data, the central server determines the speed at which the anomaly is to be negotiated, based on recent driving experience and also suggests alternate travel path which could be adopted.

24. The data mapping and publishing unit does the following function:

(a) It geographically maps the road anomaly.
(b) It creates an anomalies database which is updated from time to time based on surveys.
(c) It plots the road anomalies on a web mapping service, depending on the route selected by the user.
(d) The road anomalies thus plotted identified the name of the road as well as colour coded distress spots on the road.
(e) It suggests alternative routes which can be adopted.

25. As such, the data mapping and publishing unit creates a heat map which identifies concentration of road anomalies on various roads. This serves two functions. It helps authorities to repair the roads and correct the anomalies and also helps subscribed drivers to negotiate the anomalies and select alternate routes.

26. Thus, the entire method claimed in the subject patent involves

(a) Classification of the road anomaly, by co-relating the video of the road labelled through the data mapping system, exact geo tagging of the anomaly and average deceleration of speed while negotiating the anomaly.
(b) Tagging of the road anomalies in colour coded fashion so as to inform the authorities about the severities of the anomalies and enable them to prioritize and plan repair work.
(c) Conducting historical analysis of the deceleration through the location where the anomaly is detected and
(d) Informing subscribed users in advance of the existence and severity of road anomalies.

27. The specifications clarified that the aforesaid process is continuous and iterative so that that new spots are continuously geo-tagged, anomalies specific colours are changed from time to time and, if the road is re-surfaced or patch work is done, geo-tags are removed.

28. Thus, the process for visualization of road conditions based on specific area includes (a) collection and processing of road anomalies data for a particular area for a particular period of 10-15 days so that periodical variation in road anomalies can be noted and (b) geographical colour coding of the sub-areas within the area based on number and severity of anomalies.

29. Para 0040 of the complete specifications also refers to the process of communication of the road anomalies to the subscriber drivers/users. The user is required to switch on the web mapping service application, select the destination to which he intends to travel and press ‘Start’. When he does so, the system identifies his existing location, destination location and selects the best route1.

30. From the database of road anomalies, the system downloads the road anomaly data for the road that the driver has selected and shows it to the driver on the web mapping service application. The driver is thus warned of the upcoming road anomaly based on severity and the speed of the driver. The driver is also provided suggestion of the average speed at which the anomaly should be negotiated.

Prior Art D-2

31. The complete specifications of the prior art D-2, sans the drawings, may be reproduced thus:
(57) ABSTRACT

The present invention is directede to a road condition detection system for identifying and monitoring road conditions, and for communicating information regarding road conditions to various users. The road condition detection system is provided for capturing data indicative of road conditions and analyzing the captured data to locate and identify varius road conditions (e.g., road hazards, such as potholes, or weather conditions, such as ice). In various embodiments, the road condition detection system includes a road condition sensor array configured for being attached to a vehicle and for capturing road condition data. The captured data may be transmitted and assessed by a server configured for identifying potential road hazards or other road conditions based on the road condition data captured by the sensor array.
“ROAD CONDITION DETECTION SYSTEM

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No. 61/803,777, filed Mar. 20, 2013, the entirety of which is hereby incorporated by reference.

BACKGROUND OF THE INVENTION

Field of the Invention

Various embodiments of the present invention described herein generally relate to a road condition detection system for detecting and monitoring road conditions.

Description of Related Art

Various road conditions may make a road difficult to navigate or increase the risk of damage to a vehicle driving on the road. Some of these conditions include road imperfections (e.g., potholes, bumps, and cracks), weather conditions (e.g., wet or icy roads), low visibility (e.g., due to street lights being out), or debris in the road (e.g., tree limbs, gravel, and car accident debris). If one of these conditions is present on a given road, it would be advantageous for drivers to be notified so they may avoid a particular road hazard or hazardous road condition. Additionally, it would be advantageous for appropriate agencies, such as the local Department of Transportation, to be notified so that hazardous conditions can be addressed and the public may be apprised of current road conditions.

Several media outlets exist that inform drivers of road conditions, such as local news stations, local radio stations, and online traffic sites. Similarly, government agencies (e.g., Department of Transportation) have systems that enable drivers to report roadway conditions to the appropriate personnel. However, the above-described media outlets and agencies often depend on people manually reporting road conditions. For example, if a driver sees a hazardous condition on a given road, the driver may call the Department of Transportation and describe the condition and its location (e.g., a pothole or patch of ice). In order to address the road condition, the Department of Transportation may first send a crew to locate the condition, which may be difficult to do depending on the description given by the driver. Furthermore, information regarding the road condition may become inaccurate through the chain of communication (e.g., where one DOT employee describes the condition inaccurately to another responsible for a repair or inspection). In addition, the crew may also need to inspect the identified road condition to determine what must be done to address it. At some later point in time, a separate crew may be sent to address the condition. This process is often imprecise and inefficient, thereby resulting in lingering hazardous road conditions posing a continuing threat to drivers and vehicles.
Accordingly, there is an ongoing need in the art for systems and methods for more efficiently identifying and reporting road conditions.

BRIEF SUMMARY OF THE INVENTION

Various embodiments of the present invention are directed to a system for detecting and monitoring road conditions. According to various embodiments, the system comprises one or more memory storage areas and one or more processors in communication with the one or more memory storage areas. The processors are, collectively, configured to: receive road condition data captured by one or more road condition detection systems provided on one or more vehicles, wherein the road condition data is indicative of one or more road condition attributes of one or more road surfaces traveled by the one or more vehicles; identify, based on the received road condition data, one or more road hazards existing on the one or more road surfaces, the one or more road hazards each comprising an identified road condition that is at least potentially hazardous to a vehicle traversing the road surface; determine, based on the received road condition data, the location of the identified one or more road hazards; and store data indicative of the identified one or more road hazards and their respective locations in the one or more memory storage areas.

Various embodiments of the present invention also include a system for detecting road conditions including at least one road condition sensor array configured for being mounted to a vehicle, the road condition sensor array comprising one or more sensing devices configured capturing road condition data indicative of one or more road condition attributes of one or more road surfaces traveled by the vehicle; at least one processor configured for controlling the operation of the at least one road condition sensor array; and one or more memory storage areas configured for storing the road condition data captured by the road condition sensor array.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

Reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and wherein:
FIG. 1 is a schematic block diagram of a road condition detection system according to one embodiment of the present invention;
FIG. 2 is a schematic block diagram of road condition sensor array adapted for use on a vehicle according to one embodiment of the present invention; and
FIG. 3 is a flow diagram of steps executed by a road condition detection system according to one embodiment of the present invention; and
FIG, 4 is a flow diagram of steps executed by a central server according to another embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention now will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the inventions are shown. Indeed, these inventions may be embodied in many is different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like numbers refer to like elements throughout.

Overview

The present invention relates to systems and methods for identifying and monitoring road conditions, and for communicating information regarding road conditions to various users. According to various embodiments, a road condition detection system is provided for capturing data indicative of road conditions and analyzing the captured data to locate and identify various road conditions (e.g., road hazards, such as potholes, or weather conditions, such as ice). In various embodiments, the road condition detection system comprises a road condition sensor array configured for being attached to a vehicle and for capturing road condition data, an onboard computer for analyzing the road condition data in real time and transmitting information regarding road conditions to remotes users of the system, and a central data analysis server configured for conducting post processing analysis of the data collected by the road condition sensor array to determine additional information about various road conditions.

In various embodiments, the road condition sensor array is configured to sense and record information about a given road as the vehicle travels along the road. For example, the road condition sensor array may include an optical camera, a vibration sensor, a road surface scanner, and various other devices configured to capture road condition data indicative of various road condition attributes, such as a road’s surface profile, imperfections, illumination level, reflectivity, and/or other conditions. The road condition sensor array may also include, or may be in communication with, a geo-location device, allowing the system to geo-code the location of road condition data indicative of these attributes. According to various embodiments, a plurality of vehicles in a fleet (e.g., a fleet of delivery vehicles) may each be equipped with a road condition sensor array, thereby capturing road condition data for various roads over a wide area at various times.

In various embodiments, the road condition data collected by the road condition sensor array may be processed and stored (in whole or in part) by an onboard vehicle computer. In addition, the data capturing operations of the road condition sensor array may be controlled by the onboard computer, which may dictate when the road condition sensor array captures road condition data and the frequency with which it does so. In addition, the onboard computer may include various telematics devices and sensors monitoring dynamic attributes of the vehicle, or may be in communication with a separate telematics devices or sensors provided on the vehicle.

As described in greater detail below, the road condition data captured by the road condition sensor array may be analyzed by the onboard computer in order to identify various road conditions based on the road condition data (e.g., bumps, potholes, debris, wet or icy conditions, etc.). Information advising users of these various conditions may then be transmitted from the onboard computer to one or more users devices in real time over a network. In addition, the road condition data may be analyzed further by the central server to identify additional road conditions (e.g., more detailed information regarding cracks and potholes, visibility conditions, weather conditions, etc.). In addition, the central data analysis server may be configured to create data applied to maps indicating various road conditions (e.g., layers applied to digital maps), calculate a road condition index for individual roads or areas, and compare changes in road conditions for particular roads or locations. The analyses performed by the central server may be useful, for example, in identifying and communicating road conditions to drivers and for evaluation of road repairs and maintenance.

Road Condition Detection System

FIG. 1 shows a road condition detection system 100, according to one embodiment. In the illustrated embodiment, the road condition detection system 100 generally comprises a road condition sensor array 120 mounted on a vehicle 110, an onboard computer 130, and a central data analysis server (herein “central server”) 170. As discussed in greater detail below, the onboard computer 130 is configured to communicate with the road condition sensor array 120 in order to control the capture and storage of road condition data. The onboard computer 130 is further configured to s communicate with the central server 170 via a communication network 150 in order to transmit road condition data to the server 170 for analysis. Additionally, the central server 170 and onboard computer 130 may be configured to communicate with one or more user devices 180 (e.g., a mobile phone, tablet computer, digital information acquisition device, or the like) to provide updated road condition data to a user. The components of the illustrated embodiment are discussed in more detail below.

Road Condition Sensor Array

FIG. 2 shows a road condition sensor array 120 mounted on a vehicle 110 according to one embodiment. In the illustrated embodiment, the road condition sensor array 120 generally comprises a front sensing unit 121a housed within a detachable front mounting housing 122a, and a rear sensing unit 121b housed within a detachable rear mounting housing 122b. According to various embodiments, the front sensing unit 121a comprises various road image detection devices, including an optical camera, a vibration sensor, and a road surface scanner. As the vehicle 110 drives along a road 5, the road condition sensor array’s sensing unit 121a captures road condition data relating to the conditions of the road 5. For example, the captured road condition data may include video data, surface profile data, illumination level data, vibration data, and other data generated by other devices in the sensing unit 121a. As described in greater detail below, the sensing unit’s devices are in communication with the vehicle’s onboard computer 130, which may be configured to control the operation of the sensing unit 121a and store captured road condition data.

In various embodiments, the sensing unit 121a includes an optical camera configured to capture video and/or still images of the road surface and detect the illumination level of the road 5. For example, in one embodiment, the optical camera may be configured to capture video of the road surface continuously as the vehicle 110 travels. In such embodiments, the onboard computer 130 may continuously store video data generated by the optical camera and may be configured to buffer and transmit data to a user device 180.

According to various embodiments, the optical camera may also be configured to receive light from the road through an aperture in the camera housing, which may vary in size to control the amount of light reaching the optical sensor. The optical sensor is configured to detect the intensity of the light received through the aperture resulting in illumination level data corresponding to the illumination level of the road 5 at a given point. Accordingly, if the vehicle 110 is traveling at night along a lighted road and passes by an area where street lights are out, the illumination level data will indicate a reduction in luminous intensity. Similarly, the illumination level data may indicate poor illumination in tunnels, under bridges, or in covered areas (e.g., parking lots). In this way the optical camera is able to capture illumination level data indicative of a particular road’s varying illumination levels along the distance traveled by the vehicle 110.

In various embodiments, the optical camera may be further configured to detect the reflectivity of the surface of the road 5. This may be accomplished, for example, by the camera emitting light (e.g., via a flash bulb or LED bulb) and detecting the level of light reflected by the road surface. As such, the optical camera is able to capture reflectivity data indicative of the surface reflectivity of the road 5 at any given point. Accordingly, if a road surface is wet (e.g., due to rain or an oil leak) or coated with another hazardous substance, the reflectivity data will indicate a comparative increase in the reflectivity of the road surface. Likewise, where the road is dry, the reflectivity data will indicate a comparative decrease in the reflectivity of the road surface. In this way. the optical camera is also able to capture reflectivity data indicative of a particular road’s varying reflectivity along the distance traveled by the vehicle 110.

As will be appreciated from the description herein, the optical camera may be configured to capture video data, illumination level data, and reflectivity data, or two or more cameras may be provided, each being configured to capture illumination level data or reflectivity data. In addition, a lens may be disposed within the camera housing aperture and a shutter and/or lens cover may cover the lens when the camera is not actively recording illumination level data or reflectivity data. In this way, the shutter and/or lens cover may protect the lens from damage such as being scratched or cracked. In other embodiments, the illumination level data and reflectivity data may be captured via other suitable devices. such as laser sensors or the like.

In addition to the optical camera, the sensing unit 121a may include a road surface scanner may comprise a laser or electromagnetic sensor disposed within a scanner housing. As the vehicle travels along the road 5, the sensor is configured to scan the surface of the road 5 and capture surface profile data indicative of the road’s surface profile. Accordingly, if the vehicle 110 travels over a pothole, the captured surface profile data will indicate a depression in the road surface. Likewise. if the vehicle 110 travels over piece of debris or other object on the surface of the road 5, the captured surface profile data will indicate a protrusion on the road surface. In this way. the road surface scanner is able to capture surface profile data indicative of a particular road’s full surface profile along the distance traveled by the vehicle 110.

As noted above, the sensing unit 121a also includes a vibration sensor configured to capture vibration data indicative of the magnitude and frequency of vibration of the vehicle 110 as it travels along the road. For example, in one embodiment, the vibration sensor is configured to detect vibrations in the vehicle’s chassis (e.g., vibrations transmitted from the road surface through the wheels and suspension to the chassis). Accordingly, if the vehicle 110 travels over a pot hole, the vibration data captured by the vibration sensor will indicate a sharp change in vibration magnitude or frequency. Additionally, if the vehicle 110 is traveling a smooth road. the vibration data will indicate a low-magnitude, consistent vehicle vibration, while a rough road will result in inconsistent vibration data corresponding to various bumps and imperfections in the road surface. In this way, the vibration sensor is also able to capture vibration data indicative of the smoothness of a particular road along the distance traveled by the vehicle 110.

According to various embodiments, the sensing unit 121a may further comprise an infrared camera, a noise detecting device, and/or other road condition detecting devices. For example, the infrared camera may be used for capturing infrared data indicative of hot spots on the road surface while the noise detecting device may be used for capturing noise data indicative of loud noises associated with a vehicle traveling over a pot hole or other debris. Indeed, as will be appreciated from the description herein, the sensing unit 121a may include any road condition detecting device capable of detecting useful data indicative of one or more road conditions. Additionally, in some embodiments, the various cameras and/or sensors of the road condition sensor array 120 may have zoom capabilities in order to capture road condition data with at varying degrees of granularity.

As shown in FIG. 2, the various devices of the sensing unit 121¢ are secured within a mounting housing 122a. In the illustrated embodiment, the mounting housing 122a is mounted to the front bumper of vehicle 110 and faces the road surface directly in front of the vehicle 110 (e.g. as indicated by the dashed lines in FIG. 2). In certain embodiments, the mounting housing 122 includes a quick release mechanism configured to engage a mating member on the vehicle’s front bumper. This allows the sensing unit 121a to be easily removed from vehicle 110 and easily mounted on another vehicle. Moreover, as will be appreciated from the description herein, the vehicle 110 may include a plurality of mating members positioned at various locations on the vehicle 110 (e.g., front bumper, rear bumper, centrally underneath vehicle, side of vehicle frame, etc.) such that the one or more road condition sensor arrays 120 can be secured at various locations on the vehicle 110.

In the illustrated embodiment, the road condition sensing array 120 also includes a rear sensing unit 121b secured within a rear mounting housing 122b. According to various embodiments, the rear sensing unit 121b may comprise the same, or one or more of, the various sensors and detection devices provided in the front sensing unit 121a. In addition, the rear mounting housing 122b may be substantially the same as, or similar to, the front mounting housing 122a. As will be appreciated from FIG. 2, the provision of both front and rear sensing units 121a, 122b enables the road condition sensing array 120 to capture additional road condition data to verify various road conditions. Indeed, in certain embodiments, the rear sensing unit 121b may be configured to capture road condition data to confirm road conditions indicated by the road condition data captured by the front sensing unit 121a.

In various other embodiments, the sensing units 121a, 121b and mounting housings 122a, 122b may be provided at any suitable location on the vehicle 110 depending on its configuration and intended use. In addition, according to various embodiments, less or additional sensing units may be provided as needed. For example, in certain embodiments. only a single front or rear sensing unit may be provided. In other embodiments, additional sensing units may be placed on lateral sides of the vehicle.

Onboard Computer & Communications Network

According to various embodiments, the road condition sensor array’s sensing units 121a, 121b may be controlled by the vehicle’s onboard computer 130. In various embodiments, the onboard computer 130 comprises at least one processor, a location-determining device or sensor (e.g., a GPS sensor), a real-time clock, J-Bus protocol architecture, an electronic control module (ECM), a port for receiving data from vehicle sensors located on the vehicle 110, a communication port for receiving instruction data, a radio frequency identification (RFID) tag, a power source, a data radio for communication with a WWAN, a WLAN and/or a WPAN, a programmable logic controller (PLC), and one or more memory storage devices. The memory storage devices may include volatile memory and/or non-volatile memory, which can be embedded and/or may be removable. For example, the non-volatile memory may be embedded or removable multimedia memory cards (“MMCs”), secure digital (“SD”) memory cards. Memory Sticks, EEPROM, flash memory, hard disk, or the like. The memory storage device may also include DRAM and NVRAM memory modules. In other embodiments, various components of the onboard computer 130 (e.g., the RFID tag, the location sensor, and the PLC) may be located in the vehicle 110, external from the onboard computer 130.

The onboard computer’s location sensor may be, for example, a GPS-based sensor compatible with a low Earth orbit (LEO) satellite system, medium Earth orbit satellite system, or a Department of Defense (DOD) satellite system. Alternatively, triangulation may be used in connection with various cellular towers positioned at various locations throughout a geographic area in order to determine the location of the vehicle 110. The location sensor may be used to receive position, time, and speed data. In addition, the location sensor may be configured to detect when its vehicle 110 has entered or exited a GPS-defined geographic area (e.g., a geo-fenced area). As will be appreciated from the description herein, more than one location sensor may be utilized, and other similar techniques may likewise be used to collect geo-location information associated with the vehicle 110.

In addition, various embodiments of the onboard computer 130 may include multiple processors configured for carrying out the various processes described herein. As will be appreciated from the description herein, the onboard computer 130 may not include certain of the components described above, and may include any other suitable components in addition to, or in place of, those described above. As an example, the onboard computer 130 may include various types of communications components (e.g., to support new or improved communications techniques).

In the illustrated embodiment, the onboard computer 130 is generally configured to communicate with the road condition sensor array’s sensing units 121a, 121b in order to (i) control when the sensing units 121a, 122b capture road condition data, (ii) store the road condition data captured by the sensing units 121a, 122b, and (iii) transmit the stored road condition data to the central server 170 and/or the user device 180. For example, in one embodiment, the onboard computer 130 causes the sensing units 121a, 122b to capture road condition data continuously as the vehicle 110 travels. In other embodiments, the onboard computer 130 causes the sensing units 121a, 122b to capture road condition data at given time intervals when the vehicle 110 is on (e.g., such that all of the sensing unit’s sensors capture data every second, every 2 seconds, or every 5 seconds). In other embodiments, the onboard computer 130 causes the sensing units 121a, 122b to capture road condition data at given distance intervals as the vehicle 110 travels down road 5 (e.g., such that all of the sensing unit’s sensors capture data every 5 feet, every 10 feet, or every 50 feet traveled).

In further embodiments, the onboard computer 130 causes the sensing units 121a, 122b to start or stop capturing road condition data when the vehicle 110 changes direction, goes over a bump, or accelerates. In addition, the onboard computer 130 may be configured to monitor signals received from the sensing units 121a, 122b and capture data only when certain predefined parameters are met (e.g., illumination intensity below a predefined valued or a road surface profile deviating more than a certain amount from a predefined base profile). Moreover, the onboard computer 130 may be configured to trigger data capture by one or more specific devices in the sensing units 121a, 122b according to the criteria above. Indeed. as will be appreciated from the description herein, the onboard computer 130 may be programmed to trigger data capture by the sensing units 121a, 122b according to any desirable parameters.
As noted above, the onboard computer 130 includes a location-determining device or sensor, such as a GPS sensor, and a real-time clock. Accordingly, in various embodiments, the onboard computer 130 may be configured to associate and store location and/or date and time information—e.g., as indicated by the location sensor and clock—with the road condition data collected by the road condition sensor array 120. By associating location and date and time information with the road condition data captured by the road condition sensor array 120, the physical and temporal location of a road hazard indicated by the road condition data may b