Open Analytics Tools
Novetta is invested in Open Source Software
CLAVIN
Cartographic Location and Vicinity Indexer – Version 3.0 Now Available
An open source software package for geoparsing and georesolution. CLAVIN identifies and extracts location names from structured / unstructured text and resolves the results against a gazetteer to produce data-rich geographic entities. It natively uses Apache OpenNLP and Stanford NER to extract and resolve geospatial entities while reconciling misspellings, alternate names, and ambiguous references as part of this library. CLAVIN also integrates with Novetta’s AdaptNLP for place name extraction. It is fast, accurate, and scalable to accommodate big data in the cloud. By enriching documents with structured geodata, CLAVIN enables advanced geospatial analytics on massive volumes of text.
The MIT Center for Civic Media evaluated commercial and open source geoparsing software and found CLAVIN to provide the best combination of performance and usability, stating: Our data shows that the CLAVIN system is comparable in performance with Yahoo Placespotter and has the advantage of being free, open source and thus tunable to a news context.
CLAVIN resources:
- Novetta CLAVIN datasheet: HERE
- Spring Boot Microservice with integrated Docker container support that provides HTTP endpoints for core CLAVIN: HERE
- CLAVIN-NERD package adds support for Stanford NER integration for finding place names in text: HERE
AdaptNLP
A high level framework and library for running, training, and deploying state-of-the-art NLP models
AdaptNLP lowers the barrier to entry for practitioners and allows users ranging from beginner python coders to experienced machine learning engineers to leverage state-of-the-art NLP models and training techniques for research and production.
AdaptNLP is a python package built atop two open-source libraries: Transformers (from Hugging Face) and Flair (from Zalando Research). AdaptNLP’s unified API helps users train, fine-tune, and run pre-trained models with deep learning transformers-architecture language models like BERT, XLNet, GP2, and T5. The fine-tuning framework uses ULM-FiT for NLP tasks such as text classification, question answering, entity extraction, summarization, translation, and part-of-speech tagging.
AdaptNLP resources:
- Tutorials, guides, and class API documentation: HERE
- A GPU-compatible containerized image with AdaptNLP installed from source: HERE
- A guide to using AdaptNLP and FastAPI to stand up custom NLP models as a REST API microservice, along with a configurable containerized image: HERE
Supermicro Monitoring
A simple utility that automatically installs and configures software built by Super Micro.
Configuring and operating on-prem hardware can be time-consuming and labor-intensive. To simplify initial and ongoing hardware support for our customers, we built a simple utility, Supermicro Monitoring is a utility that automatically installs and configures software built by Super Micro. The utility monitors the health of Linux systems that use Supermicro hardware, automatically checks for active RAID controllers and hard drives, and reports health and status to support technicians. This utility fills a gap due to the lack of built-in diagnostics software for Supermicro hardware, freeing up system engineers to focus on higher-impact mission work.
Lib2nbdev
An open source package which performs a one-time conversion of existing Python libraries into nbdev format for a Jupyter Notebooks programming environment.
Adoption of nbdev can be daunting when conversion of existing Python libraries is conducted manually. Lib2nbdev generates nbdev libraries that use Jupyter Notebooks in a one-time automated conversion, allowing for a literate programming environment, conducive to CI/CD practices.
Lib2nbdev resources:
- Example settings.ini file outlining parameters needed for conversion: HERE
- Full documentation on how to use lib2nbdev: HERE
