Ebook: Data Management for Natural Scientists: A Practical Guide to Data Extraction and Storage Using Python

Data Management for Natural Scientists offers a practical guide for scientific processing of data. It covers the way from “getting hands on” experimental results to ensuring their use for addressing various scientific questions. Code snippets are provided in order to introduce the proposed workstream and to demonstrate the adjustability to specific challenges.

This book does not require in-depth previous knowledge on scripting in Python or data management, but familiarity with basic concepts such as the creation of folders and files is assumed. Some previous experience using scripting languages is certainly helpful but not mandatory for understanding the key ideas of this book and following the provided code snippets. The examples presented herein were created using Python 3 on a Windows platform.

Getting access to the data part of an experimentally obtained results file is key to all of the downstream steps in data processing. So, let’s start our Python scripting endeavour at the beginning and tackle the problem at the source. The following topics will be addressed:

– How to extract and separate data, metadata and information from an experimental results file?
– How can regular expressions assist us in this task?

Throughout this book, Python is the tool of choice for all the tasks associated with data treatment, manipulation and basic visualization. Let me explain, why this might also be the right approach for your project. On the Python homepage, it is described as a programming language that lets you work quickly and integrate systems more effectively. Anyone who has read a few lines of Python will agree that the programming project’s philosophy of focusing on code readability and readily understandable syntax has clearly been met—at least in comparison to other programming languages. To reflect this characteristic, the term pythonic has been coined. This essential feature provides a rather low entry barrier to Python: both for programming beginners and more experienced programmers aiming to engage in a new programming language. Furthermore, it is an open-source project with a large community of active users and contributors. At the time of the writing of this book, Python consistently ranks as one of the most popular programming languages.

Further benefits of Python include its versatility provided by dedicated libraries for, among others, data science, scientific computing, image processing, computer vision, web programming and scraping and many more. Also, the availability of extensive documentation is an additional aspect in favor of Python.