{ "cells": [ { "cell_type": "code", "execution_count": 1, "id": "ea2b5cac", "metadata": { "hide_input": true, "jupyter": { "source_hidden": true }, "slideshow": { "slide_type": "skip" }, "tags": [ "remove-cell" ] }, "outputs": [], "source": [ "%run -i ../python/common.py\n", "%run -i ../python/ln_preamble.py" ] }, { "cell_type": "markdown", "id": "55f80fff-ab22-4698-ba1c-779b57fe4b33", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# SLS Lecture 4 : Editors, Make and Terminal IDEs\n", "and some shell script programming to boot\n", "
\n", "\n", "https://web.archive.org/web/20110407224426/http://jtnimoy.net/workviewer.php?q=178\n", "
" ] }, { "cell_type": "markdown", "id": "5e2a4759-1816-44d2-861b-0080c0c0138f", "metadata": { "slideshow": { "slide_type": "skip" }, "tags": [] }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "303e74ea-28ac-4057-adcf-b165a774a4e0", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## RECAP" ] }, { "cell_type": "markdown", "id": "a906e88f-f680-4cb2-90c4-bbd96633a404", "metadata": { "slideshow": { "slide_type": "notes" }, "tags": [] }, "source": [ "Let review what we have discovered so far" ] }, { "cell_type": "markdown", "id": "5c5a6de2-3520-4b1b-b912-956f58f3b4d6", "metadata": { "slideshow": { "slide_type": "skip" }, "tags": [] }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "00ca7286-d82c-43c6-ad9a-602370999abd", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Computer Systems Conceptual" ] }, { "cell_type": "markdown", "id": "b7bc96a2-2296-4f2b-9c18-fba37a972b23", "metadata": {}, "source": [ "% SLIDE BODY\n", "\n", " \n", " \n", " \n", " \n", "
\n", "

\n", "Breaks down in to two parts\n", " \n", "1. Hardware\n", "2. Software \n", "

\n", " 		\n", " \n", "
\n", " " ] }, { "cell_type": "markdown", "id": "3d520910-8de2-45a6-b2a1-a0e33169f3bf", "metadata": { "slideshow": { "slide_type": "skip" }, "tags": [] }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "0a250911-7b5d-4a4a-9531-f6142fec106e", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Computer Systems Conceptual" ] }, { "cell_type": "markdown", "id": "873cbcfe-2428-40f0-9073-eebe321b2a4a", "metadata": { "tags": [] }, "source": [ "% SLIDE BODY\n", "\n", " \n", " \n", " \n", " \n", "
\n", "

\n", "Software broken down \n", "\n", "1. Operating System\n", "2. User (Everything else) \n", "

\n", " 		\n", " \n", "
\n", " " ] }, { "cell_type": "markdown", "id": "55520d17-86ea-4880-afc3-49ea45d381fd", "metadata": { "slideshow": { "slide_type": "skip" }, "tags": [] }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "c78ba5a0-bd68-430a-92ae-a9da780b31d4", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Running (Runtime) Structure of a Computer System" ] }, { "cell_type": "markdown", "id": "827df53b-66e7-47ec-a790-c60e1c64b8fe", "metadata": {}, "source": [ "
\n", "\n", "
" ] }, { "cell_type": "markdown", "id": "b726ca81-7bca-40e9-945d-5589e01c3fcd", "metadata": { "slideshow": { "slide_type": "skip" }, "tags": [] }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "e4e26550-4189-46a3-9269-82afceb999e1", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## OS Kernel - UNIX Kernel as our example" ] }, { "cell_type": "markdown", "id": "8c19fe66-a740-4163-a2bd-fc64fe45abeb", "metadata": { "tags": [] }, "source": [ "% SLIDE BODY\n", "\n", " \n", " \n", " \n", " \n", "
\n", "

\n", " \n", "### OS Kernel System Calls\n", "\n", "1. Processes talk to kernel through system calls\n", "2. System Calls can take arguments and return values\n", "

\n", " 		\n", "\n", "
\n", " " ] }, { "cell_type": "markdown", "id": "3dc15344-3257-48d8-9e54-8f13da719224", "metadata": { "slideshow": { "slide_type": "skip" }, "tags": [] }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "709e3506-d1d0-4303-a925-9987e9352a3a", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## OS Kernel - UNIX Kernel as our example" ] }, { "cell_type": "markdown", "id": "97ccbdd6-2b54-4367-b237-ecca9d1e7bd2", "metadata": { "tags": [] }, "source": [ "% SLIDE BODY\n", "\n", " \n", " \n", " \n", " \n", "
\n", "

\n", " \n", "### OS Kernel Data Structures and Objects\n", "\n", "1. Kernel provides abstract objects like processes, files, tree of path names, users, etc\n", "2. Internally, uses data structures to represent the necessary state\n", "

\n", " 		\n", "\n", "
\n", " " ] }, { "cell_type": "markdown", "id": "cfa9ad09-6646-47d8-81b0-195f53827007", "metadata": { "slideshow": { "slide_type": "skip" }, "tags": [] }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "5802b598-7aa4-4b7e-9474-39639b97d73d", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Computer Systems Practical -- UNIX Example \n", "\n", "1. ASCII Terminal provides a text based way for physically working with system\n", "2. UNIX Shells, eg. bash, provides ASCII command line \"language\"\n", "3. Core OS Ideas\n", " 1. Processes: \n", " 1. a \"running\" program\n", " 1. shell external commands: launch processes\n", " 2. signals: pause, un-pause, kill processes \n", " 3. list: `ps`\n", " 2. Paths: \n", " 1. hierarchy of names : directories and files\n", " 2. process have a working directory\n", " 3. full path vs relative path\n", " 3. I/O: Files: data and metadata\n", " - Types (typically found on all UNIXs) \n", " 1. regular files : the \"normal\" files\n", " 2. directories\n", " 3. devices files: eg. terminals (tty files), ssd's, etc\n", " 4. pipes\n", " 5. sockets\n", " 6. symbolic links\n", " - Streams and redirection\n", " 1. process open files as a stream \n", " 2. stream identified by numeric index, fd\n", " - 0: standard input\n", " - 1: standard output\n", " - 2: standard error\n", " 4. shell redirection syntax \n", " - setup streams \n", " 4. User and Group Identifiers: \n", " 1. user id 0: Root user\n", " 5. Permissions\n", " 1. File Meta data indicate owner and permissions\n", " 2. Processes have credentials\n", " 3. Accesses are permitted if process credentials and permissions match" ] }, { "cell_type": "markdown", "id": "0d6abbf8-458f-4522-ab39-8e0810cd0158", "metadata": { "slideshow": { "slide_type": "skip" }, "tags": [] }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "e426764a-052f-4465-b1fb-a26469c16e0a", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## Let's try putting it all together with an example\n", " - piping cats" ] }, { "cell_type": "markdown", "id": "857bcd85-d755-4ee3-80f7-b7ed5ae03274", "metadata": { "slideshow": { "slide_type": "skip" }, "tags": [] }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "efaff2ab-fe3e-4445-a0f9-13675063cbb5", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "In a single terminal\n", "```bash \n", "cat | cat \n", "```\n", "and illustrate" ] }, { "cell_type": "markdown", "id": "5f330221-6c93-4ac1-9ed8-03e18777245a", "metadata": { "slideshow": { "slide_type": "skip" }, "tags": [] }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "1aa2af4c-2705-4673-9e17-9b99b60c8313", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "Connect cat in two different terminals\n", "\n", "Terminal 1\n", "```bash\n", "mknod mypipe p\n", "ls -l \n", "cat > mypipe \n", "```\n", "Terminal 2\n", "```bash\n", "ls -l\n", "cat < mypipe\n", "```" ] }, { "cell_type": "markdown", "id": "df7e821c-b55d-415f-856b-8f89b4921b58", "metadata": { "slideshow": { "slide_type": "skip" }, "tags": [] }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "92207394-7d69-4fd5-8b1c-8c3968d6db8a", "metadata": { "slideshow": { "slide_type": "slide" }, "tags": [] }, "source": [ "## WHAT COMES NEXT?\n", "\n", "In order to learn to write \"real\" software we need some tools\n", "\n", "0. Shell organizing and navigating files\n", "1. ASCII Text Editors - `emacs` (or `vim` if you must)\n", "2. Build System - `make`\n", "3. UNIX as an IDE - organizing your terminal windows\n", "4. Re-vision control - `git` \n", "\n", "Then we will be able to integrate debuggers, assemblers, linkers, and compilers" ] }, { "cell_type": "markdown", "id": "8c5da214-f2f2-4455-9dcf-8b3097328e09", "metadata": { "slideshow": { "slide_type": "skip" }, "tags": [] }, "source": [ "
" ] }, { "cell_type": "markdown", "id": "6089d7f4", "metadata": { "cell_style": "split", "slideshow": { "slide_type": "slide" } }, "source": [ "## ASCII Editors and Programming\n", "\n", "- We now know what a regular File is right?\n", "- ASCII file: is simply a file who's data is encode in ASCII\n", "- Traditionally HUMAN \"friendly\" program source code has been written as ASCII files\n", " - Why?\n" ] }, { "cell_type": "markdown", "id": "5c3342ea", "metadata": { "cell_style": "split", "slideshow": { "slide_type": "-" } }, "source": [ "
\n", "\n", "
\n", "Types of Paths\n", "\n", "```\n", "b block special\n", "c character special\n", "d directory\n", "f regular file\n", "l symbolic link\n", "p FIFO\n", "s socket\n", "```" ] }, { "cell_type": "markdown", "id": "ebfcee0a", "metadata": { "cell_style": "center", "slideshow": { "slide_type": "notes" } }, "source": [ "- We now know that what we think of as a file is a bit subtle and depends on context\n", " - generally, however, as a user we think \"normal\" files -- as objects that we use to organize and store our information. Eg. File objects that are backed on devices which will hold our data until we explicitly delete it.\n", "\n", "- ASCII again is a simple coding of the English language along with some control characters\n", " - code make it easy for us to \"parse\" the contents \n", " - where each token of the coding is mapped to a single byte value\n", " \n", "- Given ASCII's simple coding it is easy to write programs that can \"process\" its contents\n", " - thus we can specify a grammar/syntax/language using ASCII. Eg. python, shell script, C, java, etc. \n", " - that we can then write programs to parse and transform into other representations that can then be \"executed\"\n", "\n", "- UNIX has many man tools for working with ASCII files" ] }, { "cell_type": "markdown", "id": "603fe6e3", "metadata": { "hide_input": true, "slideshow": { "slide_type": "slide" } }, "source": [ "## Practical side of files\n", "\n", "- We should know how to use our OS to \"manage\" files\n", " - explore what files exist and navigate the directory tree\n", " - know how to use, full and relative, paths\n", " - create and remove directories\n", " - create files, redirect a commands output into a file, and read data from files\n", " - remove files\n", " - search ASCII files\n", "- lets put all this knowledge to good use and write a program -- shell script" ] }, { "cell_type": "markdown", "id": "d095f8a2", "metadata": { "slideshow": { "slide_type": "notes" } }, "source": [ "- Navigate using `pwd`, `cd`, `ls`, using full path `ls $HOME/bin`\n", "- `mkdir`, `rmdir`, `rm -r`, the dangerous `rm -rf`\n", "- `touch`, `echo \"contents of /bin:\" > myfile`, `ls /bin >> myfile\"\n", "- `rm myfile`, `rm *.o`, \n", "- `grep`, `rgrep`, `find`\n", "- bigger example : \n", "\n", "``` bash\n", "# How can I navigate to home dir?\n", "cd ~\n", "# lets get a copy of the program source code from the internet (github)\n", "git clone https://github.com/kornia/kornia.git\n", "# use find to print all files\n", "find ~/kornia \n", "# what is the difference between * and \"*\"\n", "echo *\n", "echo \"*\"\n", "# use \"-name\" argument to tell find to restrict/filter only name that match .py\n", "find ~/kornia -name \"*.py\"\n", "# search all of them for functions in a file\n", "# What is the python syntax for defining a function?\n", "# def followed by a space\n", "# What can we use to search a file for lines that has this \"pattern\" in it?\n", "# grep\n", "grep \"def \" ~/kornia/conftest.py\n", "# need a while loop to process each file \n", "# what is read? \"help read\" -- very powerful idiom combine read with while\n", "find ~/kornia -name \"*.py\" | while read file; do grep \"defs \" $file ; done\n", "# lets turn this into a program -- shell script\n", "mkdir ~/bin\n", "echo '#!/bin/bash' > ~/bin/findfuncs\n", "echo 'find ~/kornia -name \"*.py\" | while read file; do grep \"def \" $file ; done' >> ~/bin/findfuncs\n", "cat ~/bin/findfuncs\n", "findfuncs\n", "# What are we missing?\n", "chmod +x ~/bin/findfuncs\n", "findfuncs\n", "# What are we missing?\n", "export PATH=$PATH:~/bin\n", "findfuncs\n", "```" ] }, { "cell_type": "markdown", "id": "c53dfbde", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## EDITORS\n", "\n", "- Our program has a lot to be desired of it\n", "- Pretty painful to keep going this way \n", " - although there are \"stream\" \"editors\", `sed`, `awk` and `perl` \n", "- Terminal based ASCII Editors and programming\n", " - two main UNIX Religions: VI(M) and EMACS (https://en.wikipedia.org/wiki/Emacs#Church_of_Emacs)\n", "- in reality there are many others Terminal based editors including: `nano`, `joe`, `man -k editors`\n", " - many editors allow for \"multiple panes\" and can even have terminal panes within the editor\n", " - \"pane\" divides the UI into sub-areas that can each display their own content" ] }, { "cell_type": "markdown", "id": "1d87005b", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Beyond VI and Emacs \n", "- there are more modern editors that only have a GUI\n", " - sublime, atom, vscode, kate, etc\n", " - The Jupyter Environment is actually an IDE with its own editor designed for web-browsers" ] }, { "cell_type": "markdown", "id": "e9cf1fe9", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### BUT...\n", "\n", "It really pays to know how to be functional in VIM and EMACS at least a little\n", "\n", "- it means that you can be highly productive even with just \"tty\"/terminal access.\n", " - eg. you have `ssh` terminal access to a remote computer\n", "- core component in constructing your own low-level IDE (with terminals and other tools)\n", "- understanding how things actually work and fit together\n", "- lowest common denominator is still the terminal" ] }, { "cell_type": "markdown", "id": "021022fe", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### VI and VIM\n", "\n", "- VI 1976 by Bill Joy: https://en.wikipedia.org/wiki/Vi\n", "- VIM 1991 by Bram Moolenaar: https://en.wikipedia.org/wiki/Vim_(text_editor)\n", " - VIM has in some sense subsumed VI and on most unix systems when you issue vi you really get VIM \n", " - Has some support for extensions and advanced features\n", "- Remember all these tools were written by programmers for programmers\n", " - its greatest power and its learning curve comes from its text based command interface\n", " - AKA it is not menu driven\n", " - two core modes of operation: Command mode and Insert mode\n", " - It actually has 12 modes (according to the wikipedia entry)\n", " - I only know the basics but use it very often when need to make quick changes to files\n", "- built in help `:help` and lots of online resources and tutorials -- including a `game`\n", "- Lets use VIM to improve our program" ] }, { "cell_type": "markdown", "id": "00078dd7", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### EMACS\n", "\n", "\n", "https://en.wikipedia.org/wiki/Emacs\n", "- EMACS 1976 David A. Moon and Guy L. Steele Jr\n", "- GNU EMACS ~1985 Richard Stallman \n", "- It has graphical versions but we are going to stick with the terminal mode \n", "- It is a little more recognizable as an editor as it does have menus (`F10` to access menu bar)\n", " - but it is still pretty cryptic\n", "- Its defining feature is its extensibility\n", " - no surprise it is built around a programing language. Elisp (a lisp derived language)\n", " - thousands and thousands of packages\n", " - extremely configurable\n", " - lot and lots of programmer tools and support -- often used as an IDE unto itself\n", " - generally I am going shy away from using it this way in class\n", "- all its extensibility and external packages can make emacs very complex\n", "- lots and lots of key-stroke combinations to do stuff\n", " - easiest to have a cheat sheet\n", " - use menu system and take note of keyboard shortcuts listed\n", " - learn to use complex built in help system" ] }, { "cell_type": "markdown", "id": "90564308", "metadata": { "slideshow": { "slide_type": "notes" } }, "source": [ "- I grew up on emacs\n", "- lets `cd ~/bin` so we start with this as our working directory when we are in vim\n", "- `emacs` or `emacs findfuncs`\n", "- open file \n", "- break the lines up and add a comment\n", "\n", "``` bash\n", "#!/bin/bash\n", "# this is our crude bash script to try and find all python functions\n", "# in all the .py files within a directory tree\n", "find ~/kornia -name \"*.py\" | while read file\n", "do \n", " grep \"def \" $file \n", "done\n", "```\n", "\n", "- lets add line numbers : `cat -n $file | grep def` \n", " - exit and demo `cat -n`\n", "- add to loop\n", " - start with a bug `cat-n $file`\n", " - exit run \n", "- lets start builing out our IDE: use a second terminal test \n", " - NEW Terminal \n", " - set prompt `export PS1='DEBUG$ '` to help us stay organized\n", " - ET: add `set -x`; DT: run ; ET: add `set -e`\n", " - ET: edit fix \n", " - DT: test\n", "- notes: \n", " - `set -n` : dry run good to check for syntax errors without execution\n", " - `set -u` : expansion of unset variable is an error good for catching typeos\n", "- now lets get fancier\n", "\n", "\n", "\n", "- lets extend/modify our code to produce a report of all the names of the functions we find and what files they are in \n", " - report format: one line per function definition\n", " - `:`\n", "- many ways to go about it but going to try and stick to core shell programming \n", " - and idioms I have found very useful -- learn a few well\n", "- iterate to this code:\n", " 1. add `while read line def func rest; do`\n", " 2. add `echo $func $file: $line`\n", " 3. cleanup function name with `${func%%(*}`\n", " 4. add pipe to sort `| sort`\n", "``` bash\n", "find ~/kornia -name \"*.py\" | while read file\n", "do\n", " cat -n $file | grep 'def '| while read line def func rest; do\n", " func=${func%%(*}\n", " echo $func $file: $line\n", " done\n", "done | sort\n", "```" ] }, { "cell_type": "markdown", "id": "f1aa1834", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Make\n", "\n", "- Two things go into any programming development process:\n", " - modify files \n", " - running commands that process those files\n", "- Editors are great tools for the first \n", "- but constantly running commands in the terminal is not a great solution for the second\n", " - There might be many files that we want to run commands on \n", " - There might be various different commands to run\n", " - Some commands might depend on on some file \n", " - etc." ] }, { "cell_type": "markdown", "id": "a94fd16a", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "- This is where the `make` utility comes in \n", " - make automates running commands on files when they have changed\n", " - lets a developer document what \"depends\" on what\n", " - and specify what the rules are for generating new/derived files when a dependency has changed\n", " - the most common use is in the \"building\" of programs \n", " - where various commands need to be run on the source files\n", " - to create the \"final\" program\n", " - most IDE either use `make` internal or have some equivalent\n", " - `make`, however, is a generic tool for automation\n", " " ] }, { "cell_type": "markdown", "id": "43595807", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "- Like the rest of UNIX Make is old, powerful and cryptic\n", " - 1976 Stuart Feldman https://en.wikipedia.org/wiki/Make_(software)\n", " - There are many variants we are going to focus on GNU Make\n", " - https://www.gnu.org/software/make/\n", " - Manual https://www.gnu.org/software/make/manual/make.html\n", " - Basic syntax and use is not too bad\n", " - however over the years many features and subtle syntax has been added\n", "- lets use make to automate the use of our `findfunc` command\n", " - evolving our usage and understanding as we go" ] }, { "cell_type": "markdown", "id": "4e52629f", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Make preliminaries\n", "\n", "- Lets revisit the file meta data so we can understand how make detects that something has changed\n", "- Every time a file is modified UNIX updates a field in the meta data of the file called `mtime`\n", " - every time the file is written `mtime` changes\n", " - `touch` can update the `mtime` to the current time without changing the contents\n", "- Make uses a simple idea \n", " - if file 'A' has a modification time that is newer than file 'B' \n", " - and file 'B' some how depends on file 'A' \n", " - then 'B' is considered to be out of date\n", " - any commands/rules that are needed to produce 'B' should be executed\n", "- Note Make is not really looking at the contents of the files only timestamps\n", " - touch on a 'source' will be sufficient to cause things to be rebuilt\n" ] }, { "cell_type": "markdown", "id": "13bfb9d1", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Make Basic Syntax\n", "\n", "``` make\n", "target: dependencies\n", " rules\n", "```\n", "\n", "- must use `` to indent rule lines\n", "- lets assume every time that the `CHANGELOG.md` is modified we would like to rebuild our report" ] }, { "cell_type": "code", "execution_count": 4, "id": "b61553c1", "metadata": { "hide_input": false, "jupyter": { "source_hidden": true }, "slideshow": { "slide_type": "notes" }, "tags": [ "remove-input" ] }, "outputs": [ { "data": { "text/markdown": [ "\n", "- BT lets create a project directory : findfunc\n", "- lets put script and makefile in it\n", "- start with \n", "``` make\n", "funcs.txt: ~/kornia/CHANGELOG.md\n", " ./findfuncs > $@\n", "```\n", "- lets move towards the following\n", " 1. add target `kornia/CHANGELOG.md:` no dep rule: git clone `https://github.com/kornia/kornia.git`\n", " 2. modify our findfuncs to take directory as a paramter and a usage if empty\n", " 3. progressively add Make variables\n", " 4. add a clean target and .PHONEY\n", " 5. add a funccount.txt target dep: `funcs.txt` rule: `wc -l $< > $@`\n", " 6. add `all` target with `funccount.txt` as dependency\n", " 7. add `torchfuncs.txt` target dep: `funcs.txt` rule: `grep 'torch' $< > $@`\n", "- makefile\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/markdown": [ "\n", "
\n", "\n", "\n", "``` make\n", "DIR=kornia\n", "REPO=https://github.com/kornia/kornia.git\n", "\n", ".PHONEY: clean all\n", "\n", "all: funcscount.txt torchfuncs.txt\n", "\n", "funcs.txt: ${DIR}/CHANGELOG.md\n", "\t./findfuncs ${DIR} > $@\n", "\n", "funcscount.txt: funcs.txt\n", "\twc -l $< > $@\n", "\n", "# add this after so that we can see incremental running of rules\n", "torchfuncs.txt: funcs.txt\n", "\tgrep 'torch' $< > $@\n", "\n", "kornia/CHANGELOG.md:\n", "\tgit clone ${REPO}\n", "\n", "clean:\n", "\t-rm funcs.txt funcscount.txt torchfuncs.txt\n", "\n", "distclean:\n", "\t-rm -rf ${DIR}\n", "\n", "```\n", "\n", "\n", "
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/markdown": [ "\n", "- findfuncs will need to be modified to add an argument\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/markdown": [ "\n", "
\n", "\n", "\n", "``` bash\n", "#!/bin/bash\n", "# this is our crude bash script to try and find all python functions\n", "# in all the .py files within a directory tree\n", "\n", "#set -n # -n dry run good for syntax checking (without running any commands)\n", "#set -x # tracing of simple commands on \n", "#set -e # exit script on any fail return code\n", "#set -u # error if expanding unset variable ... goood for catching typeos\n", "\n", "DIR=$1\n", "[[ -z $DIR ]] && echo \"USAGE: $0 \" && exit -1\n", "\n", "find $DIR -name \"*.py\" | while read file\n", "do\n", " cat -n $file | grep 'def '| while read line def func rest; do\n", "\tfunc=${func%%(*}\n", "\techo $func $file: $line\n", " done\t\n", "done | sort \n", "\n", "```\n", "\n", "\n", "
\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "display(Markdown(\n", " '''\n", "- BT lets create a project directory : findfunc\n", "- lets put script and makefile in it\n", "- start with \n", "``` make\n", "funcs.txt: ~/kornia/CHANGELOG.md\n", " ./findfuncs > $@\n", "```\n", "- lets move towards the following\n", " 1. add target `kornia/CHANGELOG.md:` no dep rule: git clone `https://github.com/kornia/kornia.git`\n", " 2. modify our findfuncs to take directory as a paramter and a usage if empty\n", " 3. progressively add Make variables\n", " 4. add a clean target and .PHONEY\n", " 5. add a funccount.txt target dep: `funcs.txt` rule: `wc -l $< > $@`\n", " 6. add `all` target with `funccount.txt` as dependency\n", " 7. add `torchfuncs.txt` target dep: `funcs.txt` rule: `grep 'torch' $< > $@`\n", "- makefile\n", " '''))\n", "display(Markdown(FileCodeBox(\"../src/findfuncs/Makefile\",\"make\")))\n", "display(Markdown(\n", " '''\n", "- findfuncs will need to be modified to add an argument\n", " '''))\n", "display(Markdown(FileCodeBox(\"../src/findfuncs/findfuncs\", \"bash\")))" ] } ], "metadata": { "celltoolbar": "Tags", "hide_input": true, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.5" }, "rise": { "autolaunch": false, "enable_chalkboard": true, "footer": "SLS -- Lecture 4", "header": "Unix : Editors and Make", "height": "100%", "scroll": true, "start_slideshow_at": "selected", "transition": "none", "width": "100%" } }, "nbformat": 4, "nbformat_minor": 5 }