So far in this course, we've been preparing our data with TensorFlow data sets.
Now we are going to look at how to carry out data preparation with data generators.
And this data generator class we have here, which inherits from the sequence class, is made of this
three main methods, though we could also have other methods like on Epoc.
And so let's focus on this one and understand how this could be used in preparing our data.
Now, one thing we should note is the fact that here we have an object detection problem and so we have
the image path and then the XML path.
With this image path, we expect to obtain an output tensor of shape 224 by two, 24 by three.
And then with the XML path.
So we have this with XML path, we expect to have an output tensor of shape seven by seven by 25.
So let's have this by 25.
All let's just write it out explicitly here.
So let's take this off, take this off.
And here we have.
And now we will also include the batch size.
So here we have the batch.
Now, here in this init method, we have the strain image list, which is made of the list of all trained
images from our image paths.
And then we have the strain map list, which is made of the list of all XML files.
So now you see how we obtain the image path and the XML path.
Then we specify the split size number of boxes, number of classes, and the batch size for the length
year.
We are going to define how many times or how many batches of data we are going to work with.
So if the batch size is one, then you work with the length of this image list.
So what this means essentially is if you have 20,000 images, obviously 20,000 images and 20,000 XML
files, then if the batch size is equal one, then the length you return your will be 20,000.
But if the batch size, for example, is is 20, then you have 1000.
As output length, since you would have to complete one epoch by going to the data set 1000 times since
the bar size is 20.
As you've broken out the broken down, the data set into 1000 different parts.
Now finally we have this get item method right here where we are actually going to get this output.
Now we just get it in method.
We call this data generation method, which is going to be in charge of obtaining the output.
So here we have we well, we define x and Y and then you see we have this image we read or we load the
image.
But because each and every time we are having to load this image, we actually load in a batch.
You will notice that we have an ID which is passed here.
Now this IDs go from zero right up to 1000 or 1000 minus one.
Because remember, if we have a batch size of 20 and 20,000 different images, then we are going to
have 1000 batches.
And so for the first batch, for example, we will take the index zero.
The index zero is going to be passed here.
Then we'll go from zero because this is zero times batch size.
So we'll go from 0 to 0 plus one.
That's one.
One times batch size is 20.
So we'll go from 0 to 20.
And then with this for loop will then be able to get the first 20 images and put in this x variable.
Now for the output stats for Y, we are going to call on this generate output method right here which
takes in the bound and boxes which we've gotten after pre processing the XML file and then going out
of this we have X and Y, which is what we expect.
So now define train engine and while GEN which is essentially the training and validation sets where
we have our training images and our training XML parts sent, we specify the split size number of boxes,
classes and the batch size.
And then for the training you just have to put your train generator.
And so that's it for this section.
We've just seen how to create our data set or prepare our data set using TensorFlow data generators.