hiplab/data-transport
hiplab
/
data-transport
mirror of https://github.com/lnyemba/data-transport
1
1
Fork 0
Code Issues Packages Projects Releases Wiki Activity
data-transport/notebooks/mongodb.ipynb

4.2 KiB
Raw Blame History

None <html lang="en"> <head> </head>

Writing to mongodb

Insure mongodb is actually installed on the system, The cell below creates a dataframe that will be stored within mongodb

In [1]:
#
# Writing to mongodb database
#
import transport
from transport import providers
import pandas as pd
_data = pd.DataFrame({"name":['James Bond','Steve Rogers','Steve Nyemba'],'age':[55,150,44]})
mgw = transport.factory.instance(provider=providers.MONGODB,db='demo',collection='friends',context='write')
mgw.write(_data)
print (transport.__version__)

2.0.0

Reading from mongodb

The cell below reads the data that has been written by the cell above and computes the average age within a mongodb pipeline. The code in the background executes an aggregation using db.runCommand

  • Basic read of the designated collection find=<collection>
  • Executing an aggregate pipeline against a collection aggreate=<collection>

NOTE

It is possible to use transport.factory.instance or transport.instance they are the same. It allows the maintainers to know that we used a factory design pattern.

In [4]:
import transport
from transport import providers
mgr = transport.instance(provider=providers.MONGODB,db='foo',collection='friends')
_df = mgr.read()
PIPELINE = [{"$group":{"_id":0,"_counts":{"$sum":1}, "_mean":{"$avg":"$age"}}}]
_sdf = mgr.read(aggregate='friends',pipeline=PIPELINE)
print (_df)
print ('--------- STATISTICS ------------')
print (_sdf)

           name  age
0    James Bond   55
1  Steve Rogers  150
--------- STATISTICS ------------
   _id  _counts  _mean
0    0        2  102.5

The cell bellow show the content of an auth_file, in this case if the dataset/table in question is not to be shared then you can use auth_file with information associated with the parameters.

NOTE:

The auth_file is intended to be JSON formatted

In [1]:
{
    "host":"klingon.io","port":27017,"username":"me","password":"foobar","db":"foo","collection":"friends",
    "authSource":"<authdb>","mechamism":"<SCRAM-SHA-256|MONGODB-CR|SCRAM-SHA-1>"
}
Out[1]:
{'host': 'klingon.io',
 'port': 27017,
 'username': 'me',
 'password': 'foobar',
 'db': 'foo',
 'collection': 'friends',
 'authSource': '<authdb>',
 'mechamism': '<SCRAM-SHA-256|MONGODB-CR|SCRAM-SHA-1>'}
In [ ]:
</html>