To access the "cluster’s API" server from our local machine, we only need to get the kubeconfig file created on the node during the setup. $ mkdir ~/k8s $ cd ~/k8s $ sudo cat /etc/rancher/k3s/k3s.yaml > k3s.yaml We then need to change the server key

Session 2 Let's setup a different power user on the Pi. Once your Raspberry PI 4 completed booting Raspbian, you can remotely connect to it with SSH. Open your terminal and type this command: ssh pi@raspberrypi.local This logs in the pi user. Keep in mind that the default

Read some about Luigi and how to do pipeline/workflow stuff in standard python leveraging a package like Luigi. Then use your understanding to look into how Airflow works. Luigi https://luigi.readthedocs.io/en/stable/example_top_artists.html Link luigi to airflow https://www.kdnuggets.com/2019/03/

Often times, early in cohorts, students begin to discover two things. The first is that materials they are using are incomplete, vague and contain mistakes or errors that cause confusion. The second thing is that some students discover that these revelations disturb them a great deal. One of the unfortunate

Before the OS Concepts lecture, I'd like you to watch these videos. They break down the complexities of the CPU, Memory and so on within a modern microprocessor. They move at a decent pace, and explain pretty clearly quite a few concepts you need to understand before you can really

Cellular Automata. https://bitstorm.org/gameoflife/ good sim on page, several great patterns pre-built. https://www.quantamagazine.org/john-conways-life-in-games-20150828/ bio of Conway, plus good desc of game. https://web.stanford.edu/class/sts145/Library/life.pdf bad PDF of 1970 SciAm article https://www.ibiblio.org/lifepatterns/october1970.html very

Let’s say you have a bunch of files you don't want in your repo (like .class or .ctxt files?) but you've already pushed them! (doh!) And you'd like add them to your .gitignore so that git will ignore them moving forward; these files will still be present in your

Now, let's do some python with our kafka. (Hey, your kafka and zookeeper should still be running.) (and if not, start 'em up again) in a directory named (lines10) in your projects somewhere, create a file producer.py from time import sleep from json import dumps from kafka import KafkaProducer

So make sure you have java 1.8 running $ java -version java version "1.8.0_66" Java(TM) SE Runtime Environment (build 1.8.0_66-b17) Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode) and then install kafka brew install kafka this should install

As part of your agenda, read these articles, as many as you can. You might even decide to follow one through to the end. But that's up to you. Kafka is one of the shiniest tech out there. Someplaces know it and use it, others are not sure what it

Apache Spark is a fast and general-purpose cluster computing system. It provides high-level APIs in Java, Scala, Python and R, and an optimized engine that supports general execution graphs. It also supports a rich set of higher-level tools including Spark SQL for SQL and structured data processing, MLlib for machine

Read these articles, to help cement and settle some of the things you've been learning over these last weeks. None of them are challenging, you should start to see things you recognize, and take pride in that. Sit in a comfortable spot, favorite beverage by your side, and just read.

Finding cool datasets should be something you do continuously as a Data Engineer. Ponder some of these: https://www.dataquest.io/blog/free-datasets-for-projects/ https://www.analyticsindiamag.com/popular-data-science-projects-for-aspiring-data-scientists/ https://www.springboard.com/blog/free-public-data-sets-data-science-project/

Books for Data Engineering Data Engineering Data Engineering Cookbook Plumber of DS Python Primary: Think Python Python Notes SQL Primary: Learning SQL: Master SQL Fundamentals Database Design SQL Notes PostgreSQL Notes SQL 1 SQL Style Gitlab SQL Style SQL Style Guide Clean up your SQL Style Linux/CommandLine/Bash -

Data Engineering Data Engineering Cookbook Plumber of DS Python Primary: Think Python Hitchhiker's Guide to Python Python Notes SQL Primary: Learning SQL: Master SQL Fundamentals SQL: The Complete Reference Database Design SQL Notes PostgreSQL Notes SQL 1 Really Indepth: Database Systems: The Complete Book Linux/CommandLine/Bash - Shell/Terminal

To load a missing python library into your notebook server import sys print(sys.executable) will tell you what the executable path is for your python used within your notebooks. /Users/foo/anaconda3/bin/python Then, you can do something like !/Users/foo/anaconda3/bin/python -m pip install wordcloud