{"id":773,"date":"2025-03-28T21:38:32","date_gmt":"2025-03-28T21:38:32","guid":{"rendered":"https:\/\/techtrendfeed.com\/?p=773"},"modified":"2025-03-28T21:38:32","modified_gmt":"2025-03-28T21:38:32","slug":"constructing-tensorflow-pipelines-with-vertex-ai","status":"publish","type":"post","link":"https:\/\/techtrendfeed.com\/?p=773","title":{"rendered":"Constructing TensorFlow Pipelines with Vertex AI"},"content":{"rendered":"<p> <br \/>\n<\/p>\n<div id=\"article-start\">\n<p>How will you guarantee your machine studying fashions get the high-quality information they should thrive? In at the moment\u2019s <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.analyticsvidhya.com\/machine-learning\/\" target=\"_blank\" rel=\"noreferrer noopener\">machine studying<\/a> panorama, dealing with information effectively is as essential as constructing robust fashions. Feeding high-quality, well-structured information into your fashions can considerably affect efficiency and coaching velocity. The TensorFlow Dataset API simplifies this course of by providing set of instruments to construct, handle, and optimize information pipelines. On this information, we\u2019ll go step-by-step from configuring your growth setting utilizing Vertex AI Workbench to loading information from numerous sources and incorporating these pipelines into your mannequin coaching course of.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-learning-objectives\">Studying Aims<\/h3>\n<ul class=\"wp-block-list\">\n<li>Construct datasets from in-memory arrays in addition to exterior information sources similar to CSV and TFRecord information.<\/li>\n<li>Make the most of operations similar to mapping, shuffling, batching, caching, and prefetching to streamline information processing.<\/li>\n<li>Seamlessly incorporate your datasets into TensorFlow\u2019s mannequin coaching routines for environment friendly mannequin growth.<\/li>\n<li>Be taught to launch a Vertex AI Workbench occasion, arrange a Jupyter Pocket book, and begin working.<\/li>\n<li>Improve your machine studying fashions by making use of information augmentation methods immediately in your information pipelines.<\/li>\n<\/ul>\n<p><em><strong>This text was revealed as part of the\u00a0<\/strong><\/em><a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.analyticsvidhya.com\/datahack\/blogathon\" target=\"_blank\" rel=\"noreferrer noopener\"><em><strong>Information Science Blogathon.<\/strong><\/em><\/a><\/p>\n<h2 class=\"wp-block-heading\" id=\"h-what-is-tensorflow\">What&#8217;s TensorFlow?<\/h2>\n<p><a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.analyticsvidhya.com\/blog\/2021\/11\/tensorflow-for-beginners-with-examples-and-python-implementation\/\" target=\"_blank\" rel=\"noopener\">TensorFlow<\/a> is an open-source platform developed by Google for machine studying and <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.analyticsvidhya.com\/blog\/2021\/12\/a-guide-on-deep-learning-from-basics-to-advanced-concepts\/\" target=\"_blank\" rel=\"noreferrer noopener\">deep studying <\/a>analysis. It gives an in depth ecosystem of instruments and libraries, permitting researchers to push the boundaries of what\u2019s potential in machine studying and enabling builders to construct and deploy clever purposes with ease. TensorFlow helps each high-level APIs (like Keras) and low-level operations, making it accessible for newcomers whereas remaining highly effective for superior customers.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-what-is-vertex-ai-workbench\">What&#8217;s Vertex AI Workbench?<\/h2>\n<p>Vertex AI Workbench is a managed growth setting in Google Cloud that&#8217;s designed that can assist you construct and prepare machine studying fashions. It gives a totally managed Jupyter Pocket book expertise together with preinstalled machine studying libraries, together with TensorFlow and PyTorch. With <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/cloud.google.com\/vertex-ai-notebooks?hl=en\" target=\"_blank\" rel=\"nofollow noopener\">Vertex AI Workbench<\/a>, you&#8217;ll be able to seamlessly combine your native growth with cloud computing sources, making it simpler to work on large-scale tasks with out worrying about infrastructure setup.<\/p>\n<p>On this information, not solely will you learn to work with TensorFlow\u2019s Dataset API, however additionally, you will see how one can arrange your setting utilizing Vertex AI Workbench. We are going to cowl every part from launching a brand new occasion, making a Jupyter Pocket book, and loading the datasets.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-understanding-the-tensorflow-dataset-api\">Understanding the TensorFlow Dataset API<\/h2>\n<p>The <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.tensorflow.org\/api_docs\/python\/tf\/data\/Dataset\" target=\"_blank\" rel=\"nofollow noopener\">TensorFlow Dataset API<\/a> is a set of instruments designed to simplify the method of constructing information enter pipelines. In any machine studying activity, your mannequin\u2019s efficiency relies upon not simply on the algorithm itself but in addition on the standard and circulate of the information being fed into it. The Dataset API lets you carry out duties like loading information, preprocessing it, and remodeling it on the go.<\/p>\n<p>What makes this API so highly effective is its potential to chain a number of operations in a single, easy-to-understand sequence. You may load information from numerous sources, apply obligatory transformations (similar to scaling or normalization), and even shuffle the information to stop the mannequin from overfitting. This strategy not solely makes your code cleaner and simpler to take care of, but it surely additionally optimizes efficiency by leveraging methods like caching and prefetching.<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-setting-up-your-environment-with-vertex-ai-workbench\">Setting Up Your Setting with Vertex AI Workbench<\/h2>\n<p>Earlier than you begin working with the TensorFlow Dataset API, you want a sturdy setting. Vertex AI Workbench is a superb alternative for this goal as a result of it gives a totally managed, cloud-based growth setting that comes with all of the instruments you want pre-installed.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-launch-vertex-ai-workbench-instance\">Launch Vertex AI Workbench Occasion<\/h3>\n<ul class=\"wp-block-list\">\n<li>Begin by logging into your Google Cloud account. From the Navigation menu, search and choose Vertex AI.<\/li>\n<\/ul>\n<figure class=\"wp-block-image size-full is-resized figure  mt-2 mb-2 d-table mx-auto\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1142\" height=\"633\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/LAUNCH-VERTEX.webp\" alt=\"LAUNCH VERTEX\" class=\"wp-image-228070\" style=\"width:682px;height:auto\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/LAUNCH-VERTEX.webp 1142w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/LAUNCH-VERTEX-300x166.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/LAUNCH-VERTEX-768x426.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/LAUNCH-VERTEX-150x83.webp 150w\" sizes=\"(max-width: 1142px) 100vw, 1142px\"\/><\/figure>\n<ul class=\"wp-block-list\">\n<li>Click on on the \u201cAllow All Really useful APIs\u201d button. This ensures that your mission has entry to all the required API providers.<\/li>\n<li>Within the navigation menu, click on on Workbench. Ensure you are within the Cases view.<\/li>\n<\/ul>\n<figure class=\"wp-block-image size-full is-resized figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1530\" height=\"733\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Workbench.webp\" alt=\"Workbench.\" class=\"wp-image-228071\" style=\"width:743px;height:auto\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Workbench.webp 1530w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Workbench-300x144.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Workbench-768x368.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Workbench-150x72.webp 150w\" sizes=\"auto, (max-width: 1530px) 100vw, 1530px\"\/><\/figure>\n<ul class=\"wp-block-list\">\n<li>Click on on Create New to launch a brand new Workbench occasion. You can be prompted to configure the occasion:\n<ul class=\"wp-block-list\">\n<li><b>Title:<\/b> Give your occasion a significant identify, similar to lab-workbench.<\/li>\n<li><strong>Area and Zone:<\/strong> Choose the suitable area and zone the place you need your occasion to be positioned.<\/li>\n<li><b>Superior Choices:<\/b> If wanted, customise the occasion settings by deciding on choices like machine sort or disk measurement.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul class=\"wp-block-list\">\n<li>After configuration, click on Create. It&#8217;d take a couple of minutes in your occasion to be arrange. As soon as it&#8217;s prepared, you will note a inexperienced checkmark subsequent to its identify.<\/li>\n<\/ul>\n<figure class=\"wp-block-image size-full is-resized figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"510\" height=\"674\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/new-instance.webp\" alt=\"new instance\" class=\"wp-image-228072\" style=\"width:473px;height:auto\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/new-instance.webp 510w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/new-instance-227x300.webp 227w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/new-instance-150x198.webp 150w\" sizes=\"auto, (max-width: 510px) 100vw, 510px\"\/><\/figure>\n<figure class=\"wp-block-image size-full is-resized figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1222\" height=\"583\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/instances.webp\" alt=\"instances\" class=\"wp-image-228073\" style=\"width:758px;height:auto\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/instances.webp 1222w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/instances-300x143.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/instances-768x366.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/instances-150x72.webp 150w\" sizes=\"auto, (max-width: 1222px) 100vw, 1222px\"\/><\/figure>\n<ul class=\"wp-block-list\">\n<li>Click on Open JupyterLab subsequent to your occasion\u2019s identify. It will open the Jupyter Lab interface in a brand new tab in your browser.<\/li>\n<\/ul>\n<figure class=\"wp-block-image size-full figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1917\" height=\"887\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/lab-workbench.webp\" alt=\"lab workbench\" class=\"wp-image-228074\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/lab-workbench.webp 1917w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/lab-workbench-300x139.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/lab-workbench-768x355.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/lab-workbench-1536x711.webp 1536w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/lab-workbench-150x69.webp 150w\" sizes=\"auto, (max-width: 1917px) 100vw, 1917px\"\/><\/figure>\n<h3 class=\"wp-block-heading\" id=\"h-creating-a-jupyter-notebook\">Making a Jupyter Pocket book<\/h3>\n<p>After getting your JupyterLab interface open, you can begin a brand new Python Pocket book by clicking on the Python 3 icon. It\u2019s a good suggestion to rename the pocket book to one thing descriptive. To do that, right-click on the file identify (which could initially be Untitled.ipynb) and choose Rename Pocket book. Select a reputation that displays the mission, similar to \u201cnew_project\u201d. Additionally change the kernel from python 3 to TensorFlow 2-11 (Native).<\/p>\n<figure class=\"wp-block-image size-full figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1821\" height=\"816\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/creating-jupyter-notebook-1.webp\" alt=\"creating jupyter notebook; TensorFlow Dataset API on Google Cloud\" class=\"wp-image-228076\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/creating-jupyter-notebook-1.webp 1821w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/creating-jupyter-notebook-1-300x134.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/creating-jupyter-notebook-1-768x344.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/creating-jupyter-notebook-1-1536x688.webp 1536w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/creating-jupyter-notebook-1-150x67.webp 150w\" sizes=\"auto, (max-width: 1821px) 100vw, 1821px\"\/><\/figure>\n<h3 class=\"wp-block-heading\" id=\"h-manipulate-data-with-tf-data\">Manipulate information with tf.information<\/h3>\n<p>First add the <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/github.com\/kponna\/tensorflow-dataset-api\/blob\/main\/taxi-train.csv\" target=\"_blank\" rel=\"nofollow noopener\">taxi-train.csv<\/a> and <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/github.com\/kponna\/tensorflow-dataset-api\/blob\/main\/taxi-valid.csv\" target=\"_blank\" rel=\"nofollow noopener\">taxi-valid.csv<\/a> dataset into the pocket book.<\/p>\n<figure class=\"wp-block-image size-full figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"403\" height=\"494\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Manipulate-data-with-tf.data_.webp\" alt=\"Manipulate data with tf.data; TensorFlow Dataset API on Google Cloud\" class=\"wp-image-228077\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Manipulate-data-with-tf.data_.webp 403w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Manipulate-data-with-tf.data_-245x300.webp 245w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Manipulate-data-with-tf.data_-150x184.webp 150w\" sizes=\"auto, (max-width: 403px) 100vw, 403px\"\/><\/figure>\n<h4 class=\"wp-block-heading\" id=\"h-importing-the-required-libraries\">Importing the Required Libraries<\/h4>\n<p>First, we have to import TensorFlow and NumPy, after which set the TensorFlow logging stage to a minimal setting. This reduces log verbosity throughout execution.<\/p>\n<pre class=\"wp-block-code\"><code>import tensorflow as tf\nimport numpy as np\n\nprint(\"TensorFlow model:\", tf.model.VERSION)\n\n# Set minimal TF logging stage.\nimport os\nos.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'<\/code><\/pre>\n<h3 class=\"wp-block-heading\" id=\"h-creating-a-dataset-from-memory\">Making a Dataset from Reminiscence<\/h3>\n<p>As soon as your setting is about up, you can begin working with information. The only approach to start is by making a dataset from reminiscence. This implies changing information saved in your pc\u2019s reminiscence (like lists or NumPy arrays) right into a format that TensorFlow can course of.<\/p>\n<p>Think about you&#8217;ve a small set of numbers that you simply need to use for a fundamental experiment. The TensorFlow Dataset API lets you shortly convert these numbers right into a dataset that may be manipulated additional. This course of is easy and could be prolonged to extra complicated information buildings.<\/p>\n<p>For instance, you would possibly begin with a easy NumPy array that accommodates a number of numbers. Utilizing the Dataset API, you&#8217;ll be able to create a dataset from this array. The dataset can then be iterated over, and you may apply numerous transformations similar to mapping a perform to every factor.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-creating-the-synthetic-dataset\">Creating the Artificial Dataset<\/h3>\n<p>We first create an artificial dataset. On this instance, we generate our characteristic vector X and a corresponding label vector Y utilizing the linear equation y=2x+10.<\/p>\n<pre class=\"wp-block-code\"><code>N_POINTS = 10\nX = tf.fixed(vary(N_POINTS), dtype=tf.float32)\nY = 2 * X + 10 <\/code><\/pre>\n<p>Subsequent, we outline a perform that accepts our characteristic and label arrays, together with the variety of coaching passes (epochs) and the specified batch measurement. This perform constructs a TensorFlow Dataset by slicing the tensors, repeating them for the required variety of epochs, and batching them (dropping any remaining examples to maintain batch sizes constant).<\/p>\n<pre class=\"wp-block-code\"><code>def make_synthetic_dataset(X, Y, epochs, batch_size):\n    # Create the dataset from tensor slices\n    ds = tf.information.Dataset.from_tensor_slices((X, Y))\n    # Repeat the dataset and batch it (drop the rest for consistency)\n    ds = ds.repeat(epochs).batch(batch_size, drop_remainder=True)\n    return ds    <\/code><\/pre>\n<p>Let\u2019s check our perform by iterating twice over our dataset in batches of three datapoints:<\/p>\n<pre class=\"wp-block-code\"><code>BATCH_SIZE = 3\nEPOCHS = 2\n\ndataset = make_synthetic_dataset(X, Y, epochs=EPOCHS, batch_size=BATCH_SIZE)\n\nprint(\"Artificial dataset batches:\")\nfor i, (x_batch, y_batch) in enumerate(dataset):\n    print(f\"Batch {i}: x: {x_batch.numpy()}  y: {y_batch.numpy()}\")\n    assert len(x_batch) == BATCH_SIZE\n    assert len(y_batch) == BATCH_SIZE<\/code><\/pre>\n<figure class=\"wp-block-image size-full figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"373\" height=\"146\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/synthetic-database.webp\" alt=\"synthetic database\" class=\"wp-image-228078\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/synthetic-database.webp 373w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/synthetic-database-300x117.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/synthetic-database-150x59.webp 150w\" sizes=\"auto, (max-width: 373px) 100vw, 373px\"\/><\/figure>\n<h2 class=\"wp-block-heading\" id=\"h-loss-function-and-gradient-computation\">Loss Operate and Gradient Computation<\/h2>\n<p>Subsequent, we outline the imply squared error (MSE) loss perform and a helper perform to compute gradients. These features stay much like our earlier implementation.<\/p>\n<pre class=\"wp-block-code\"><code>def loss_mse(X, Y, w0, w1):\n    Y_pred = w0 * X + w1\n    error = (Y_pred - Y) ** 2\n    return tf.reduce_mean(error)\n\ndef compute_gradients(X, Y, w0, w1):\n    with tf.GradientTape() as tape:\n        current_loss = loss_mse(X, Y, w0, w1)\n    return tape.gradient(current_loss, [w0, w1]), current_loss<\/code><\/pre>\n<h3 class=\"wp-block-heading\" id=\"h-training-loop\">Coaching loop<\/h3>\n<p>Now, we replace our coaching loop in order that it iterates over the tf.information.Dataset created by our perform. On this instance, we prepare the mannequin over 250 epochs utilizing a batch measurement of two.<\/p>\n<p>First, initialize the mannequin parameters as TensorFlow variables:<\/p>\n<pre class=\"wp-block-code\"><code># Initialize mannequin parameters\nw0 = tf.Variable(0.0)\nw1 = tf.Variable(0.0)\n\nEPOCHS_TRAIN = 250\nBATCH_SIZE_TRAIN = 2\nLEARNING_RATE = 0.02\n\n# Create the coaching dataset (artificial)\ntrain_dataset = make_synthetic_dataset(X, Y, epochs=EPOCHS_TRAIN, batch_size=BATCH_SIZE_TRAIN)\n<\/code><\/pre>\n<p>Then, we run the coaching loop utilizing stochastic gradient descent. The loop updates the mannequin parameters with every batch, and we print the coaching standing each 100 steps.<\/p>\n<pre class=\"wp-block-code\"><code># Coaching loop\nprint(\"nStarting coaching loop for artificial linear regression:\")\nMSG = \"Step {step} - loss: {loss:.6f}, w0: {w0:.6f}, w1: {w1:.6f}\"\nfor step, (X_batch, Y_batch) in enumerate(train_dataset):\n    grads, loss_val = compute_gradients(X_batch, Y_batch, w0, w1)\n    # Replace the parameters utilizing gradient descent\n    w0.assign_sub(LEARNING_RATE * grads[0])\n    w1.assign_sub(LEARNING_RATE * grads[1])\n    \n    if step % 100 == 0:\n        print(MSG.format(step=step, loss=loss_val.numpy(), w0=w0.numpy(), w1=w1.numpy()))\n\n# Closing assertions (tolerance based mostly)\nassert loss_val &lt; 1e-6\nassert abs(w0.numpy() - 2) &lt; 1e-3\nassert abs(w1.numpy() - 10) &lt; 1e-3<\/code><\/pre>\n<figure class=\"wp-block-image size-full figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"505\" height=\"292\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/training-loop.webp\" alt=\"training loop\" class=\"wp-image-228079\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/training-loop.webp 505w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/training-loop-300x173.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/training-loop-150x87.webp 150w\" sizes=\"auto, (max-width: 505px) 100vw, 505px\"\/><\/figure>\n<h2 class=\"wp-block-heading\" id=\"h-loading-data-from-disk\">Loading Information from Disk<\/h2>\n<p>In sensible purposes, information is commonly saved on disk reasonably than in reminiscence. Loading information from disk with these strategies ensures that you would be able to deal with giant datasets effectively and put together them for mannequin coaching. Two widespread codecs for storing information are CSV and TFRecord.\u00a0<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-loading-a-csv-file\">Loading a CSV File<\/h3>\n<p>CSV (Comma-Separated Values) information are broadly used for storing tabular information. The TensorFlow Dataset API gives a handy approach to learn CSV information. The method includes parsing every line of the file to transform textual content into numeric information, batching the outcomes, and making use of any further transformations.<\/p>\n<p>Beneath, we outline the column names and default values for our CSV file:<\/p>\n<pre class=\"wp-block-code\"><code>CSV_COLUMNS = [\n    'fare_amount',\n    'pickup_datetime',\n    'pickup_longitude',\n    'pickup_latitude',\n    'dropoff_longitude',\n    'dropoff_latitude',\n    'passenger_count',\n    'key'\n]\nLABEL_COLUMN = 'fare_amount'\nDEFAULTS = [[0.0], ['na'], [0.0], [0.0], [0.0], [0.0], [0.0], ['na']]<\/code><\/pre>\n<p>Subsequent, we wrap the CSV dataset creation right into a perform that reads the information based mostly on a file sample and a specified batch measurement:<\/p>\n<pre class=\"wp-block-code\"><code>def make_csv_dataset(sample, batch_size):\n    # Create dataset from CSV information with specified column names and defaults.\n    ds = tf.information.experimental.make_csv_dataset(\n        file_pattern=sample,\n        batch_size=batch_size,\n        column_names=CSV_COLUMNS,\n        column_defaults=DEFAULTS,\n        header=True\n    )\n    return ds \n# For demonstration, assume the CSV information are positioned in '..\/toy_data\/'.\ntemp_ds = make_csv_dataset('taxi-train.csv', batch_size=2)\nprint(\"nSample CSV dataset (prefetched):\")\nprint(temp_ds)<\/code><\/pre>\n<figure class=\"wp-block-image size-full figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1148\" height=\"130\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/csv-dataset.webp\" alt=\"csv dataset ; TensorFlow Dataset API on Google Cloud\" class=\"wp-image-228080\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/csv-dataset.webp 1148w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/csv-dataset-300x34.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/csv-dataset-768x87.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/csv-dataset-150x17.webp 150w\" sizes=\"auto, (max-width: 1148px) 100vw, 1148px\"\/><\/figure>\n<p>To enhance readability, let\u2019s iterate over the primary two components of this dataset and convert them into commonplace Python dictionaries:<\/p>\n<pre class=\"wp-block-code\"><code>for information in temp_ds.take(2):\n    print({ok: v.numpy() for ok, v in information.gadgets()})\n    print(\"n\")<\/code><\/pre>\n<figure class=\"wp-block-image size-full figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1136\" height=\"301\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/dataa.webp\" alt=\"TensorFlow Dataset API on Google Cloud\" class=\"wp-image-228081\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/dataa.webp 1136w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/dataa-300x79.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/dataa-768x203.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/dataa-150x40.webp 150w\" sizes=\"auto, (max-width: 1136px) 100vw, 1136px\"\/><\/figure>\n<h3 class=\"wp-block-heading\" id=\"h-loading-a-tfrecord-file\">Loading a TFRecord File<\/h3>\n<p>TFRecord is a binary format optimized for TensorFlow. It permits quicker studying speeds in comparison with CSV information and is very environment friendly for big datasets. Whereas the code supplied right here focuses on CSV, related methods could be utilized when working with TFRecord information.<\/p>\n<p>For instance:<\/p>\n<pre class=\"wp-block-code\"><code>def parse_tfrecord(example_proto):\n    # Outline the options anticipated within the TFRecord\n    feature_description = {\n        'feature1': tf.io.FixedLenFeature([], tf.float32),\n        'feature2': tf.io.FixedLenFeature([], tf.float32)\n    }\n    return tf.io.parse_single_example(example_proto, feature_description)\n\n# Create a dataset from a TFRecord file\ntfrecord_dataset = tf.information.TFRecordDataset(\"information\/sample_data.tfrecord\")\ntfrecord_dataset = tfrecord_dataset.map(parse_tfrecord)\ntfrecord_dataset = tfrecord_dataset.batch(4)\n\n# Iterate by way of the TFRecord dataset\nfor batch in tfrecord_dataset:\n    print(batch)\n<\/code><\/pre>\n<h2 class=\"wp-block-heading\" id=\"h-transforming-datasets-mapping-batching-and-shuffling\">Reworking Datasets: Mapping, Batching, and Shuffling<\/h2>\n<p>After getting created your dataset, the subsequent step is to remodel it. Transformation is a broad time period that covers a number of operations:<\/p>\n<ul class=\"wp-block-list\">\n<li><b>Mapping:<\/b> This operation applies a particular perform to each factor within the dataset. For instance, you would multiply each quantity by two or carry out extra complicated mathematical operations.<\/li>\n<li><b>Shuffling:<\/b> Shuffling the dataset is essential as a result of it randomizes the order of the information. Randomization helps stop your mannequin from studying any biases associated to the order of the information, which might enhance the generalization of your mannequin.<\/li>\n<li><b>Batching:<\/b> Batching includes grouping your information into smaller chunks. As an alternative of feeding particular person information factors to your mannequin, batching lets you course of a number of information factors directly, which might result in extra environment friendly coaching.<\/li>\n<\/ul>\n<p>For our taxi dataset, we need to separate the options from the label (fare_amount). We additionally need to take away undesirable columns like pickup_datetime and key.<\/p>\n<pre class=\"wp-block-code\"><code># Specify columns that we are not looking for in our characteristic dictionary.\nUNWANTED_COLS = ['pickup_datetime', 'key']\n\ndef extract_features_and_label(row):\n    # Extract the label (fare_amount)\n    label = row[LABEL_COLUMN]\n    # Create a options dictionary by copying the row and eradicating undesirable columns and the label\n    options = row.copy()\n    options.pop(LABEL_COLUMN)\n    for col in UNWANTED_COLS:\n        options.pop(col, None)\n    return options, label<\/code><\/pre>\n<p>We will check our perform by iterating over a number of examples from our CSV dataset:<\/p>\n<pre class=\"wp-block-code\"><code>for row in temp_ds.take(2):\n    options, label = extract_features_and_label(row)\n    print(options)\n    print(label, \"n\")\n    assert UNWANTED_COLS[0] not in options.keys()\n    assert UNWANTED_COLS[1] not in options.keys() <\/code><\/pre>\n<figure class=\"wp-block-image size-full figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1128\" height=\"286\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/ordered-dict.webp\" alt=\"TensorFlow Dataset API on Google Cloud\" class=\"wp-image-228082\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/ordered-dict.webp 1128w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/ordered-dict-300x76.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/ordered-dict-768x195.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/ordered-dict-150x38.webp 150w\" sizes=\"auto, (max-width: 1128px) 100vw, 1128px\"\/><\/figure>\n<h3 class=\"wp-block-heading\" id=\"h-batching-the-data\">Batching the Information<\/h3>\n<p>We will refine our dataset creation course of by incorporating batching and making use of our feature-label extraction perform. This helps in forming information batches which can be immediately consumable by the coaching loop.<\/p>\n<pre class=\"wp-block-code\"><code>def create_dataset(sample, batch_size):\n# The tf.information.experimental.make_csv_dataset() methodology reads CSV information right into a dataset\n    dataset = tf.information.experimental.make_csv_dataset(\n        sample, batch_size, CSV_COLUMNS, DEFAULTS)\n    return dataset.map(extract_features_and_label)\n    \nBATCH_SIZE = 2\n\ntemp_ds = create_dataset('taxi-train.csv', batch_size=2)\n\nfor X_batch, Y_batch in temp_ds.take(2):\n    print({ok: v.numpy() for ok, v in X_batch.gadgets()})\n    print(Y_batch.numpy(), \"n\")\n    assert len(Y_batch) == BATCH_SIZE<\/code><\/pre>\n<figure class=\"wp-block-image size-full figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1136\" height=\"201\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Batching-the-Data.webp\" alt=\"Batching the Data; TensorFlow Dataset API on Google Cloud\" class=\"wp-image-228083\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Batching-the-Data.webp 1136w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Batching-the-Data-300x53.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Batching-the-Data-768x136.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Batching-the-Data-150x27.webp 150w\" sizes=\"auto, (max-width: 1136px) 100vw, 1136px\"\/><\/figure>\n<h3 class=\"wp-block-heading\" id=\"h-shuffling-and-prefetching-for-efficient-training\">Shuffling and Prefetching for Environment friendly Coaching<\/h3>\n<p>When coaching a deep studying mannequin, it&#8217;s essential to shuffle your information in order that totally different employees course of numerous elements of the dataset concurrently. Moreover, prefetching information helps overlap the information loading course of with mannequin coaching, bettering general effectivity.<\/p>\n<p>We will lengthen our dataset creation perform to incorporate shuffling, caching, and prefetching. We introduce a mode parameter to distinguish between coaching (which requires shuffling and repeating) and analysis (which doesn&#8217;t).<\/p>\n<pre class=\"wp-block-code\"><code>def build_csv_pipeline(sample, batch_size=1, mode=\"eval\"):\n    ds = tf.information.experimental.make_csv_dataset(\n        file_pattern=sample,\n        batch_size=batch_size,\n        column_names=CSV_COLUMNS,\n        column_defaults=DEFAULTS,\n        header=True\n    )\n    # Map every row to (options, label)\n    ds = ds.map(extract_features_and_label)\n    # Cache the dataset to enhance velocity if studying from disk repeatedly.\n    ds = ds.cache()\n    \n    if mode == 'prepare':\n        # Shuffle with a buffer measurement (right here, arbitrarily utilizing 1000) and repeat indefinitely.\n        ds = ds.shuffle(buffer_size=1000).repeat()\n    \n    # Prefetch the subsequent batch (AUTOTUNE makes use of optimum settings)\n    ds = ds.prefetch(tf.information.AUTOTUNE)\n    return ds\n    \n# Testing the pipeline in coaching mode\nprint(\"nSample batch from coaching pipeline:\")\ntrain_ds = build_csv_pipeline('taxi-train.csv', batch_size=2, mode=\"prepare\")\nfor options, label in train_ds.take(1):\n    print({ok: v.numpy() for ok, v in options.gadgets()})\n    print(\"Label:\", label.numpy())\n\n# Testing the pipeline in analysis mode\nprint(\"nSample batch from analysis pipeline:\")\neval_ds = build_csv_pipeline('taxi-valid.csv', batch_size=2, mode=\"eval\")\nfor options, label in eval_ds.take(1):\n    print({ok: v.numpy() for ok, v in options.gadgets()})\n    print(\"Label:\", label.numpy())<\/code><\/pre>\n<figure class=\"wp-block-image size-full figure  mt-2 mb-2 d-table mx-auto\"><img loading=\"lazy\" decoding=\"async\" width=\"1134\" height=\"232\" src=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Shuffling-and-Prefetching-for-Efficient-Training.webp\" alt=\"Shuffling and Prefetching for Efficient Training\" class=\"wp-image-228084\" srcset=\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Shuffling-and-Prefetching-for-Efficient-Training.webp 1134w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Shuffling-and-Prefetching-for-Efficient-Training-300x61.webp 300w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Shuffling-and-Prefetching-for-Efficient-Training-768x157.webp 768w, https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/03\/Shuffling-and-Prefetching-for-Efficient-Training-150x31.webp 150w\" sizes=\"auto, (max-width: 1134px) 100vw, 1134px\"\/><\/figure>\n<h2 class=\"wp-block-heading\" id=\"h-data-augmentation-and-advanced-techniques\">Information Augmentation and Superior Strategies<\/h2>\n<p>Information augmentation is a vital approach in deep studying, notably in domains like picture processing. The Dataset API lets you combine augmentation immediately into your pipeline. For instance, should you want to add random noise to your dataset:<\/p>\n<pre class=\"wp-block-code\"><code>def augment_data(x):\n    return x + tf.random.uniform([], -0.5, 0.5)\n\n# Apply information augmentation\naugmented_dataset = dataset.map(augment_data)<\/code><\/pre>\n<p>This step will increase the range of your information, serving to your mannequin generalize higher throughout coaching.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-optimizing-your-data-pipeline\">Optimizing Your Information Pipeline<\/h3>\n<p>To additional improve efficiency, think about using caching and prefetching methods. Caching saves the state of your processed dataset in reminiscence or on disk, whereas prefetching overlaps information preparation with mannequin execution:<\/p>\n<pre class=\"wp-block-code\"><code>optimized_dataset = dataset.cache().shuffle(100).batch(32).prefetch(tf.information.AUTOTUNE)\n<\/code><\/pre>\n<h2 class=\"wp-block-heading\" id=\"h-best-practices-for-production-pipelines\">Greatest Practices for Manufacturing Pipelines<\/h2>\n<p>When shifting from experimentation to manufacturing, think about the next finest practices:<\/p>\n<ul class=\"wp-block-list\">\n<li><b>Modular Pipeline Design:<\/b> Break down your pipeline into small, reusable features.<\/li>\n<li><b>Sturdy Error Dealing with:<\/b> Implement mechanisms to gracefully deal with corrupt or lacking information.<\/li>\n<li><b>Scalability Testing:<\/b> Validate your pipeline with small subsets of knowledge earlier than scaling to bigger datasets.<\/li>\n<li><b>Efficiency Monitoring:<\/b> Repeatedly observe your pipeline\u2019s efficiency to establish and tackle potential bottlenecks.<\/li>\n<\/ul>\n<p>By following these pointers, you&#8217;ll be able to make sure that your information pipelines stay environment friendly and dependable, even underneath heavy manufacturing hundreds.<\/p>\n<p>You&#8217;ll find the pocket book and the outputs within the hyperlink \u2013 <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/github.com\/kponna\/tensorflow-dataset-api\/blob\/main\/new_project.ipynb\" target=\"_blank\" rel=\"nofollow noopener\">right here<\/a>.<\/p>\n<p>References:\u00a0<a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/github.com\/GoogleCloudPlatform\/training-data-analyst\" target=\"_blank\" rel=\"nofollow noopener\">Google Cloud Platform\u2019s repository<\/a>\u00a0<\/p>\n<h2 class=\"wp-block-heading\" id=\"h-conclusion\">Conclusion<\/h2>\n<p>The TensorFlow Dataset API is a elementary part in creating environment friendly and scalable machine studying pipelines. On this information, we began by updating our linear regression instance to make use of a TensorFlow Dataset created in reminiscence. We then demonstrated how one can load information from disk, notably CSV information, and defined how one can rework, batch, and shuffle information for each coaching and analysis.<\/p>\n<p>On this information, we explored how one can construct and optimize information pipelines utilizing the TensorFlow Dataset API. Beginning with artificial information generated in reminiscence, we walked by way of creating datasets, making use of transformations, and integrating these pipelines into coaching loops. We additionally lined sensible methods for loading information from disk, notably CSV information, and demonstrated how one can incorporate shuffling, caching, and prefetching to spice up efficiency.<\/p>\n<p>Through the use of features to extract options and labels, batch information, and construct strong pipelines with shuffling, caching, and prefetching, you&#8217;ll be able to streamline the information ingestion course of in your machine studying fashions. These methods not solely simplify your code but in addition improve mannequin efficiency by making certain that the information is fed effectively into the coaching loop.<\/p>\n<h3 class=\"wp-block-heading\" id=\"h-key-takeaways\">Key Takeaways<\/h3>\n<ul class=\"wp-block-list\">\n<li><strong>Environment friendly information dealing with is vital<\/strong>: TensorFlow Dataset API streamlines information pipelines for higher mannequin efficiency.<\/li>\n<li><strong>Vertex AI Workbench simplifies ML growth<\/strong>: A managed Jupyter Pocket book setting with preinstalled ML libraries.<\/li>\n<li><strong>Optimize information loading<\/strong>: Use operations like batching, caching, and prefetching to boost coaching effectivity.<\/li>\n<li><strong>Seamless mannequin integration<\/strong>: Simply incorporate datasets into TensorFlow coaching routines.<\/li>\n<li><strong>Information augmentation boosts ML fashions<\/strong>: Improve coaching datasets with transformation methods for improved accuracy.<\/li>\n<\/ul>\n<h2 class=\"wp-block-heading\" id=\"h-frequently-asked-questions\">Regularly Requested Questions<\/h2>\n<div class=\"schema-faq wp-block-yoast-faq-block\">\n<div class=\"schema-faq-section\" id=\"faq-question-1742889655647\"><strong class=\"schema-faq-question\">Q1. What&#8217;s the TensorFlow Dataset API?<\/strong> <\/p>\n<p class=\"schema-faq-answer\">A. The TensorFlow Dataset API is a set of instruments that assist effectively construct, handle, and optimize information pipelines for machine studying fashions.<\/p>\n<\/p><\/div>\n<div class=\"schema-faq-section\" id=\"faq-question-1742889675342\"><strong class=\"schema-faq-question\">Q2. Why is information dealing with essential in machine studying?<\/strong> <\/p>\n<p class=\"schema-faq-answer\">A. Nicely-structured and high-quality information improves mannequin accuracy, coaching velocity, and general efficiency.<\/p>\n<\/p><\/div>\n<div class=\"schema-faq-section\" id=\"faq-question-1742889712502\"><strong class=\"schema-faq-question\">Q3. What&#8217;s Vertex AI Workbench?<\/strong> <\/p>\n<p class=\"schema-faq-answer\">A. Vertex AI Workbench is a managed Jupyter Pocket book setting on Google Cloud for creating and coaching ML fashions.<\/p>\n<\/p><\/div>\n<div class=\"schema-faq-section\" id=\"faq-question-1742889730827\"><strong class=\"schema-faq-question\">This fall. How does the TensorFlow Dataset API enhance information processing?<\/strong> <\/p>\n<p class=\"schema-faq-answer\">A. It permits operations like mapping, shuffling, batching, caching, and prefetching to streamline information circulate.<\/p>\n<\/p><\/div>\n<div class=\"schema-faq-section\" id=\"faq-question-1742889752753\"><strong class=\"schema-faq-question\">Q5. What are the advantages of utilizing Vertex AI Workbench for TensorFlow?<\/strong> <\/p>\n<p class=\"schema-faq-answer\">A. It gives a totally managed, cloud-based growth setting with preinstalled ML libraries and seamless cloud integration.<\/p>\n<\/p><\/div>\n<div class=\"schema-faq-section\" id=\"faq-question-1742889807054\"><strong class=\"schema-faq-question\">Q6. How can I create a TensorFlow dataset from reminiscence?<\/strong> <\/p>\n<p class=\"schema-faq-answer\">A. Use <code>tf.information.Dataset.from_tensor_slices()<\/code> to transform NumPy arrays or lists right into a TensorFlow dataset.<\/p>\n<\/p><\/div><\/div>\n<p><strong>The media proven on this article isn&#8217;t owned by Analytics Vidhya and is used on the Writer\u2019s discretion.<\/strong><\/p>\n<div class=\"border-top py-3 author-info my-4\">\n<div class=\"author-card d-flex align-items-center\">\n<div class=\"flex-shrink-0 overflow-hidden\">\n                                    <a rel=\"nofollow\" target=\"_blank\" href=\"https:\/\/www.analyticsvidhya.com\/blog\/author\/krishnaveni140696\/\" class=\"text-decoration-none active-avatar\"><br \/>\n                                                                       <img decoding=\"async\" src=\"https:\/\/av-eks-lekhak.s3.amazonaws.com\/media\/lekhak-profile-images\/converted_image_uFajTEe.webp\" width=\"48\" height=\"48\" alt=\"Krishnaveni Ponna\" loading=\"lazy\" class=\"rounded-circle\"\/><\/p>\n<p>                                <\/a>\n                                <\/div><\/div>\n<p>Hi there! I am a passionate AI and Machine Studying fanatic at the moment exploring the thrilling realms of Deep Studying, MLOps, and Generative AI. I get pleasure from diving into new tasks and uncovering modern methods that push the boundaries of know-how. I will be sharing guides, tutorials, and mission insights based mostly by myself experiences, so we will be taught and develop collectively. Be a part of me on this journey as we discover, experiment, and construct wonderful options on the planet of AI and past!<\/p>\n<\/p><\/div><\/div>\n<p><h4 class=\"fs-24 text-dark\">Login to proceed studying and luxuriate in expert-curated content material.<\/h4>\n<p>                        <button class=\"btn btn-primary mx-auto d-table\" data-bs-toggle=\"modal\" data-bs-target=\"#loginModal\" id=\"readMoreBtn\">Preserve Studying for Free<\/button>\n                    <\/p>\n\n","protected":false},"excerpt":{"rendered":"<p>How will you guarantee your machine studying fashions get the high-quality information they should thrive? In at the moment\u2019s machine studying panorama, dealing with information effectively is as essential as constructing robust fashions. Feeding high-quality, well-structured information into your fashions can considerably affect efficiency and coaching velocity. The TensorFlow Dataset API simplifies this course of [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":775,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[55],"tags":[475,477,476,478],"class_list":["post-773","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-machine-learning","tag-building","tag-pipelines","tag-tensorflow","tag-vertex"],"_links":{"self":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/773","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=773"}],"version-history":[{"count":1,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/773\/revisions"}],"predecessor-version":[{"id":774,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/posts\/773\/revisions\/774"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=\/wp\/v2\/media\/775"}],"wp:attachment":[{"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=773"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=773"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techtrendfeed.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=773"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}<!-- This website is optimized by Airlift. Learn more: https://airlift.net. Template:. Learn more: https://airlift.net. Template: 69d9690a190636c2e0989534. Config Timestamp: 2026-04-10 21:18:02 UTC, Cached Timestamp: 2026-06-13 20:07:09 UTC -->