In this lecture we are going to do a few more tests in CO lab.
Specifically we're going to look at some ways to upload your own data set to collab.
Let's say for example you're a client or employer gives you a CSP file or you downloaded a CSP from
Kaggle.
How can we then make this file accessible from our call lab notebook.
In this lecture we are going to discuss a few different ways of doing this.
The first method we're going to look at is just to use the classic Linux command w get as mentioned
previously you can run command line commands by preceding the command with the bang symbol or exclamation
mark.
So let's go ahead and download the arrhythmia dataset.
Now we want to check where the data went so let's use it bang l s to see if the data is in our current
directory
OK.
It looks like it is now let's use the head command to see the first few lines of the data file and also
to check whether or not the file has a header row.
So it looks like it does not have a header row
next let's try to load in the data using pandas.
We're going to pass in a header equals none since we know that the data does not have a header
next since the data has many columns.
We're just going to take the first few.
We're also going to rename the columns because they're currently just integer values as usual since
this data is from the UCI machine learning repository.
You can just check the documentation if you want to know more about the data like what each column is
so let's run this
next let's create a histogram of these data columns since notebook by default makes the plot pretty
small.
We're going to import map plot lib and change the figure size once we've done that.
We can call DFA dot hist to create histogram for each column.
Note that I've added a semicolon to the end of the F dot hist because if you don't then no book will
print out the last returned value like it usually does which we don't want right now.
So here are some nice histogram is for you to look at.
Next let's create a common plot for data analysis.
The scatter matrix.
This does a scatter plot between each feature and every other feature along the diagonal it just plots
a histogram of each feature which we've already seen
all right.
So pretty standard so far
next let's look at the second method of loading in data which also applies when you have a you are out.
This is to use tends to flow directly specifically the cars get file function.
Now you might be thinking why would we use tensor flow if this course is about pi to work.
That's a great question and I would understand if you were hesitant.
However it's important to realize that these are all just Python libraries.
You're not restricted to using only one or the other.
That would be silly.
As an example there are several libraries for making HDTV requests.
One is called requests.
Another is called Your relic.
I don't think anyone would hesitate to use both in their Python code if you need more convincing.
It's actually common.
Even if you're working in PI talk to make use of the text pre processing functionality that comes with
characters.
So in fact data scientists and machine learning engineers have already been doing this for years.
Don't think just because I'm using pi to which I'm not allowed to use the utilities from any other deep
learning library let's start by assigning the euro to a variable called Euro.
We're going to be using the auto SPG data set although it doesn't really matter what you use for this
example.
As long as you can access it directly via your
let's run this next we're going to make sure we have tensor flow 2.0 installed so we're going to run
pip install tensor flow and then prints out the version to make sure that we have the correct one
note that when I first created this lecture tensor flow to was not yet released today you could just
do import sensor flow SDF and it'll automatically load the latest release which should be 2.0 or higher.
In addition remember that this uses the caris API so an equivalent alternative to this would be to just
use carries itself whenever you see CFD KRS you can replace that with just the careless.
So TFT carries that get file becomes crossed start get file presuming you imported carries earlier although
remember that Kerry's installation still require a back end.
So ultimately you would probably already have tens of low installed next we're going to call the carers
get filed function.
The first argument is the file path.
We want to save too.
And the second argument is the far source so let's run this note that it's possible to save the file
to a different directory but we'll be saving it to Chris's default folder so you can see from the printout
that the file ends up in slash root slash scare us slash datasets next let's call the head command so
that we can see the first few lines of the file
as you can see it's not exactly a C as V.
Instead each column is separated by whitespace and there is no hetero so in order to load this data
we can still use the pen as read C as V function but we have to pass in two extra arguments the first
argument is to say that there's no hetero so hetero equals none.
And the second extra argument is the tell partners that the delimiter is whitespace.
So we set the limb whitespace equal to true
X. We call DFT head is to make sure everything works as expected.
So as you can see the result appears to be in the right format and from here you can process this data
using Python code as you normally would.
The third method we're going to look at in order to add your own files to collab is to upload the file
directly in order to do this.
We have to run a special collab function
so we say from Google that collab import files then we call files dot upload so let's run this so you
see that this creates an upload button which we can click and then choose a file from the local file
system.
Some of the truce daily minimum temperatures
and if we prints out the returned value you can see that it's a dictionary where the file name is the
key and the value is the file contents
if we use the command bang unless we can see that the file has been uploaded to the working directory
next let's read in the file using pandas to make sure we get what we expect now this file has some garbage
lines near the end.
So I've accounted for that by setting the argument error bad lines equal to False.
This ignores errors but prints them out as they are encountered
as you can see the file is loaded in successfully
to follow up this example we're going to look at a variation on what we just did.
You recall that when you're writing code in Python sometimes it's useful to split your code amongst
several files.
This helps to organize your code and keep similar things all in one place while keeping different things
separate as a simple example.
Sometimes we'll learn about multiple algorithms in one chorus but we'll test all those algorithms on
the same dataset.
So there's no point in rewriting the code to load in the data set multiple different times.
Instead we can write the data loading code once and then import it from each file.
Now you might wonder is since we're working in code lab how can you import a function from a Python
script.
If that python script is on your local harddrive luckily we can take the same approach we already have
been to upload that file to Google collab.
So here I'm going to call fouls that upload again and this time I'm uploading the python script.
Fake you till that pie
so fake you tell that pi contains only one function called my useful function and all it does is print
out hello world.
So once you've uploaded the file you can see that we can import it just like we would if we were working
locally.
So I can say from fake you tell.
Import my useful function.
Then when I call my useful function you can see that hello world is printed out just like we expect
and by the way you might be wondering as I did what the path of the current directory actually is to
determine this.
You can just run the usual Linux command Pete WD and that prints out
slash content so slash content is our current working directory
the last thing I want to cover is something you're probably all wondering.
Google Drive is for storing files.
So is it possible to access files on your google drive.
And of course the answer is yes.
So in order to do this we have to import drive from Google at then we have to mount the drive by calling
drive down Mount and specifying the path slash content slash G drive.
So this is going to give you an authorization code.
So you go to the euro in your browser
it asks you to sign in as a accepts terms and then it gives you a code.
You copy this code and you put it back into this box.
You hit Enter
Okay so that works.
So after we've done this we can call LSI again to check what's now in the current directory
we can see that there is now an extra thing here.
She drive.
So let's allez G drive and see what that gives us.
All right.
So it looks like we now have a thing called Google Drive.
Once again I'll ask this.
And remember that you have to add quotes if your path contains whitespace
and now we can see a bunch of files that are in my google drive which is essentially a bunch of VIP
content for the VIP versions of my courses.