PDF to network graph in seconds
NC Data now can instantly create a network graph of the content of a PDF file. Actually, it can plot the network graph of the content of up to 5 PDF files. Just upload up to 5 PDF files and you can get the network graph of people, companies, organizations, concepts discussed in the document(s) in seconds.
No user interaction is needed, except for the click of a button. The graph gets iterated after the first visualization, so clicking on the button again will provide a deeper analysis and so on. I saw that on the third iteration the graph becomes insightful, as shown in this video.
The important part here is that there is only one api call, which is to parse the initial extracted text and return the entities and relationships. This is done by inducing the AI to return the response in a specific format (and this is the really challenging part). The api response is parsed in the browser, so every graph iteration later is done in the browser, which means much less tokens are used.
The video here shows a IEA report on oil. The first visualization does not provide anything interesting. The second one, which is done by simply clicking the same button again, shows some more information. The third click shows an insightful graph. This type of reports is challenging, because entities and relationships may not be easily defined, but the application does a nice job and successfully understands it. For other types of documents, where entities and relationships are more easily identifiable, the application provides helpful network graphs right after the first click or the second. But still, it is a matter of seconds.
In general, this works with PDF files containing any types of information and the critical bit is the initial extraction of entities and relationships, which, since it is done by "inducing" a response in a specific format may not be always reliable. The flip side is that this is the evidence that models can be adapted to specific needs and become very solid and reliable when fed with a structure they expect (while at the same time keeping their generative upside as opposed to a response from a database). But even without knowing the content structure, the application is anyway pretty solid and reliable. There may be edge cases where the model does not effectively extract entities and relationships (plus it does not do OCR), but if users have documents of the same type, the model can be made much stronger, reliable and efficient for a user specific need.
Here's the video.
Enjoy!