initial push

data/000.txt

@@ -0,0 +1,5 @@
+<frame >
+        
+        <titlepage>
+        
+        </frame>

data/001reminder.txt

@@ -0,0 +1,14 @@
+<frame title="reminder">
+        <list>
+        <e>No Case Study next week</e>
+<l2st>
+<e>neither Tuesday (29.11) nor Thursday (01.12)</e>
+<e>if you need help: just write me an email!</e>
+</l2st>
+<e>In two weeks: Case Study switched</e>
+<l2st>
+<e>Q+A Tuesday (6.12, 14:00) online only</e>
+<e>Case Study Meeting Thursday (08.12, 14:00-16:00), in OH12 Room 3.032</e>
+</l2st>
+        </list>
+        </frame>

data/002Big Picture.txt

@@ -0,0 +1,20 @@
+<frame title="Big Picture">
+        <list>
+        <e>Goal for this case study: Have better hyperparameters than pyod!</e>
+<e>So each of you: Gets assigned two algorithms!</e>
+<l2st>
+<e>One fairly simple before</e>
+<e>One more complicated one today</e>
+</l2st>
+<e>Try to find the best possible hyperparameters for your algorithms</e>
+<l2st>
+<e>Try to be clever (for example: PCA: $n_{components}<n_{features}$. Maybe $\frac{n_{components}}{n_{features}}$ constant?</e>
+</l2st>
+<e>Afterwards</e>
+<l2st>
+<e>Write down your findings into a simple function (given data, what are my best hyperparameters)</e>
+<e>Write down your finding into a report (together, double collumn. 6 Pages per student, plus comparison of algorithms to each other)</e>
+<e>One final presentation together in front of my colleagues. About 10min per student.</e>
+</l2st>
+        </list>
+        </frame>

data/003Evaluating your hyperparameter.txt

@@ -0,0 +1,23 @@
+<frame title="Evaluating your hyperparameter">
+        <list>
+        <e>My suggestion: Compare to normal parameters.</e>
+<e>This means you get two lists of AUC scores</e>
+<e>Your params: [0.80,0.75,0.73,....,0.95]</e>
+<e>Pyod params: [0.82,0.71,0.48,....,0.95]</e>
+<e>look at two values</e>
+<e>$\sum_i your_i-pyod_i$</e>
+<l2st>
+<e>Total improvment. If positive, then your parameters help;)</e>
+<e>But hard to see if this is significant    </e>
+</l2st>
+<e>Fraction of $your_i>pyod_i$</e>
+<l2st>
+<e>Quantised, so does not care about improving your parameters further</e>
+<e>But easy to see if this is significant</e>
+<l3st>
+<e>0.5->Probably just random</e>
+<e>0.9->Probably quite significant</e>
+</l3st>
+</l2st>
+        </list>
+        </frame>

data/004How to continue.txt

@@ -0,0 +1,9 @@
+<frame title="How to continue">
+        <list>
+        <e>See how far you can improve this?</e>
+<e>Treat this as a supervised optimisation problem: Given this dataset, find the best hyperparameters</e>
+<e>Might be useful to look at more input parameters</e>
+<e>Might help to formulate your parameters differently</e>
+<e>But be aware of \textbf{overfitting}!</e>
+        </list>
+        </frame>

data/005Intro to Deep Learning.txt

@@ -0,0 +1,3 @@
+<frame title="Intro to Deep Learning">
+<i f="..//prep/05Intro_to_Deep_Learning/adsasda.png" wmode="True"></i>
+</frame>

data/006Intro to Deep Learning.txt

@@ -0,0 +1,3 @@
+<frame title="Intro to Deep Learning">
+<i f="..//prep/06Intro_to_Deep_Learning/adsasd.png" wmode="True"></i>
+</frame>

data/007Intro to Deep Learning.txt

@@ -0,0 +1,17 @@
+<frame title="Intro to Deep Learning">
+        <list>
+        <e>The idea is always the same:</e>
+<l2st>
+<e>Define complicated model to learn (often millions of parameters)</e>
+<e>Define loss function that this model should minimize (example: $\sum_i (y_i-f(x_i))^2$)</e>
+<e>Find parameters that minimize the loss (->Backpropagation)</e>
+</l2st>
+<e>Usually Neural Networks:</e>
+<l2st>
+<e>$f(x)=f_n(x)=activation(A_n\cdot f_{n-1}(x)+b_n)$</e>
+<e>$f_0(x)=x$</e>
+</l2st>
+<e>Powerful, as you can show that when there are 3 Layers+ (and infinitely sized matrices), you can approximate any function</e>
+<e>->So a model becomes a loss function</e>
+        </list>
+        </frame>

data/008Autoencoder.txt

@@ -0,0 +1,3 @@
+<frame title="Autoencoder">
+<i f="..//prep/08Autoencoder/ae.png" wmode="True"></i>
+</frame>

data/009Autoencoder.txt

@@ -0,0 +1,28 @@
+<frame title="Autoencoder">
+        <list>
+        <e>Lets look at some of its Hyperparameters</e>
+<e>Autoencoder Specific</e>
+<l2st>
+<e>Compression factor (Latent space size)</e>
+<e>Loss function (mse?)</e>
+</l2st>
+<e>Neural Network architecture</e>
+<l2st>
+<e>Number of layers</e>
+<e>Number of neurons in each layer (Shape of the matrices $A_n$)</e>
+</l2st>
+<e>Optimisation parameters</e>
+<l2st>
+<e>Learning Rate</e>
+<l3st>
+<e>Controls how fast the parameters are found</e>
+<e>To high value makes the training unstable</e>
+</l3st>
+<e>Batch size</e>
+<l3st>
+<e>Controls how many samples are averaged together.</e>
+<e>Lower values make the training more stable, but also the result less optimal</e>
+</l3st>
+</l2st>
+        </list>
+        </frame>

data/010For next time.txt

@@ -0,0 +1,8 @@
+<frame title="For next time">
+        <list>
+        <e>(if you have not finished finding good parameters for your old algorithm, continue searching for them)</e>
+<e>Take a look at your new algorithm</e>
+<e>Run it once on cardio, take a look at which parameters you have</e>
+<e>Prepare a similar presentation to last time (include your cardio result)</e>
+        </list>
+        </frame>

general.txt

@@ -0,0 +1,12 @@
+<plt>
+
+<name Current experiment status>
+<title Lets go deeper!>
+<stitle Anomaly Detection and AutoML>
+
+<institute ls9 tu Dortmund>
+
+<theme CambridgeUS>
+<colo dolphin>
+
+</plt>

old/01Anomaly_Detection/q

@@ -0,0 +1,6 @@
+Two distributions
+<l2st>
+One known (=normal)
+One unknown (=anomalies)
+</l2st>
+Seperate them

old/02Anomaly_Detection/q

@@ -0,0 +1,7 @@
+Two distributions
+<l2st>
+One known (=normal)
+One unknown (=anomalies)
+</l2st>
+Seperate them
+Problem: few anomalies

old/03Anomaly_Detection/q

@@ -0,0 +1,8 @@
+Anomalies are rare, so often only a few datapoints known (e.g. Machine Failure in an Aircraft)
+In practice, anomalies might appear that are not known during testing
+->So train the model only on normal samples
+Unsupervised Machine Learning
+<l2st>
+What can we say without knowing anomalies?
+''Understand you dataset''
+</l2st>

old/04Anomaly_Detection/q

@@ -0,0 +1,8 @@
+Anomalies are rare, so often only a few datapoints known (e.g. Machine Failure in an Aircraft)
+In practice, anomalies might appear that are not known during testing
+->So train the model only on normal samples
+Unsupervised Machine Learning
+<l2st>
+What can we say without knowing anomalies?
+''Understand you dataset''
+</l2st>

old/05Anomaly_Detection/q

@@ -0,0 +1,6 @@
+Seems easy? Now do this
+<l2st>
+in thousands of dimensions
+with complicated distributions
+and overlap between anomalies and normal points
+</l2st>

old/06AutoML/q

@@ -0,0 +1,3 @@
+Most machine learning requires Hyperparameter Optimisation
+(Find model parameters that result in the best results)
+->AutoML: Do this automatically as fast as possible

old/07AutoAD/q

@@ -0,0 +1,9 @@
+So lets combine both (Auto Anomaly Detection)
+->Problem
+<l2st>
+AutoMl requires Evaluation (loss, accuracy, AUC) to optimize
+AD can only be evaluated with regards to the anomalies
+->no longer unsupervised
+</l2st>
+So most Anomaly Detection is ''unoptimized''
+

old/08Solution_1_Metrics/q

@@ -0,0 +1,3 @@
+So how to solve this?
+One option: Think of some function to evaluate only the normal points
+->A bit hard to do in a case study

old/08Solution_2_ZeroShot_Learning/q

@@ -0,0 +1,5 @@
+So how to solve this?
+One option: ''Just find the best solution directly''
+->Zero Shot AutoML
+Find best practices for hyperparameters
+Requires optimisation for each model seperately -> matches the case study structure quite well!

old/09Course/q

@@ -0,0 +1,12 @@
+Basics of Scientific Computing
+Basics of AD
+Basics of AutoML
+Build groups for each algorithm
+<l2st>
+Choose a set of Hyperparameters
+Find ''best practice`s'' for them
+Maybe consider more complicated Transformations (Preprocessing, Ensemble)
+</l2st>
+Compare between groups (best algorithm for current situation)
+Evaluate on new datasets
+Write a report/Present your work

old/10Questions/q

@@ -0,0 +1,7 @@
+Requirements:
+<l2st>
+MD Req 1->MD Req 8
+Basic Python/Math Knowledge
+Motivation to learn something new;)
+</l2st>
+Registration till Saturday, by Email to Simon.Kluettermann@cs.tu-dortmund.de

out/compile.bat

@@ -0,0 +1,3 @@
+pdflatex main.tex
+pdflatex main.tex
+

out/compile.sh

@@ -0,0 +1,3 @@
+pdflatex main.tex
+pdflatex main.tex
+

out/label.json

@@ -0,0 +1,29 @@
+[
+  {
+    "typ": "img",
+    "files": [
+      "..//prep/05Intro_to_Deep_Learning/adsasda.png"
+    ],
+    "label": "prep05Intro_to_Deep_Learningadsasdapng",
+    "caption": "",
+    "where": "../case3/data/005Intro to Deep Learning.txt"
+  },
+  {
+    "typ": "img",
+    "files": [
+      "..//prep/06Intro_to_Deep_Learning/adsasd.png"
+    ],
+    "label": "prep06Intro_to_Deep_Learningadsasdpng",
+    "caption": "",
+    "where": "../case3/data/006Intro to Deep Learning.txt"
+  },
+  {
+    "typ": "img",
+    "files": [
+      "..//prep/08Autoencoder/ae.png"
+    ],
+    "label": "prep08Autoencoderaepng",
+    "caption": "",
+    "where": "../case3/data/008Autoencoder.txt"
+  }
+]

out/main.aux

@@ -0,0 +1,92 @@
+\relax 
+\providecommand\hyper@newdestlabel[2]{}
+\providecommand\HyperFirstAtBeginDocument{\AtBeginDocument}
+\HyperFirstAtBeginDocument{\ifx\hyper@anchor\@undefined
+\global\let\oldcontentsline\contentsline
+\gdef\contentsline#1#2#3#4{\oldcontentsline{#1}{#2}{#3}}
+\global\let\oldnewlabel\newlabel
+\gdef\newlabel#1#2{\newlabelxx{#1}#2}
+\gdef\newlabelxx#1#2#3#4#5#6{\oldnewlabel{#1}{{#2}{#3}}}
+\AtEndDocument{\ifx\hyper@anchor\@undefined
+\let\contentsline\oldcontentsline
+\let\newlabel\oldnewlabel
+\fi}
+\fi}
+\global\let\hyper@last\relax 
+\gdef\HyperFirstAtBeginDocument#1{#1}
+\providecommand\HyField@AuxAddToFields[1]{}
+\providecommand\HyField@AuxAddToCoFields[2]{}
+\@writefile{nav}{\headcommand {\slideentry {0}{0}{1}{1/1}{}{0}}}
+\@writefile{nav}{\headcommand {\beamer@framepages {1}{1}}}
+\newlabel{reminder<1>}{{2}{2}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {reminder<1>}{2}}
+\newlabel{reminder}{{2}{2}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {reminder}{2}}
+\@writefile{nav}{\headcommand {\slideentry {0}{0}{2}{2/2}{}{0}}}
+\@writefile{nav}{\headcommand {\beamer@framepages {2}{2}}}
+\newlabel{Big Picture<1>}{{3}{3}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Big Picture<1>}{3}}
+\newlabel{Big Picture}{{3}{3}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Big Picture}{3}}
+\@writefile{nav}{\headcommand {\slideentry {0}{0}{3}{3/3}{}{0}}}
+\@writefile{nav}{\headcommand {\beamer@framepages {3}{3}}}
+\newlabel{Evaluating your hyperparameter<1>}{{4}{4}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Evaluating your hyperparameter<1>}{4}}
+\newlabel{Evaluating your hyperparameter}{{4}{4}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Evaluating your hyperparameter}{4}}
+\@writefile{nav}{\headcommand {\slideentry {0}{0}{4}{4/4}{}{0}}}
+\@writefile{nav}{\headcommand {\beamer@framepages {4}{4}}}
+\newlabel{How to continue<1>}{{5}{5}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {How to continue<1>}{5}}
+\newlabel{How to continue}{{5}{5}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {How to continue}{5}}
+\@writefile{nav}{\headcommand {\slideentry {0}{0}{5}{5/5}{}{0}}}
+\@writefile{nav}{\headcommand {\beamer@framepages {5}{5}}}
+\newlabel{Intro to Deep Learning<1>}{{6}{6}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Intro to Deep Learning<1>}{6}}
+\newlabel{Intro to Deep Learning}{{6}{6}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Intro to Deep Learning}{6}}
+\newlabel{fig:prep05Intro_to_Deep_Learningadsasdapng}{{6}{6}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {fig:prep05Intro_to_Deep_Learningadsasdapng}{6}}
+\@writefile{nav}{\headcommand {\slideentry {0}{0}{6}{6/6}{}{0}}}
+\@writefile{nav}{\headcommand {\beamer@framepages {6}{6}}}
+\newlabel{Intro to Deep Learning<1>}{{7}{7}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Intro to Deep Learning<1>}{7}}
+\newlabel{Intro to Deep Learning}{{7}{7}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Intro to Deep Learning}{7}}
+\newlabel{fig:prep06Intro_to_Deep_Learningadsasdpng}{{7}{7}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {fig:prep06Intro_to_Deep_Learningadsasdpng}{7}}
+\@writefile{nav}{\headcommand {\slideentry {0}{0}{7}{7/7}{}{0}}}
+\@writefile{nav}{\headcommand {\beamer@framepages {7}{7}}}
+\newlabel{Intro to Deep Learning<1>}{{8}{8}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Intro to Deep Learning<1>}{8}}
+\newlabel{Intro to Deep Learning}{{8}{8}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Intro to Deep Learning}{8}}
+\@writefile{nav}{\headcommand {\slideentry {0}{0}{8}{8/8}{}{0}}}
+\@writefile{nav}{\headcommand {\beamer@framepages {8}{8}}}
+\newlabel{Autoencoder<1>}{{9}{9}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Autoencoder<1>}{9}}
+\newlabel{Autoencoder}{{9}{9}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Autoencoder}{9}}
+\newlabel{fig:prep08Autoencoderaepng}{{9}{9}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {fig:prep08Autoencoderaepng}{9}}
+\@writefile{nav}{\headcommand {\slideentry {0}{0}{9}{9/9}{}{0}}}
+\@writefile{nav}{\headcommand {\beamer@framepages {9}{9}}}
+\newlabel{Autoencoder<1>}{{10}{10}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Autoencoder<1>}{10}}
+\newlabel{Autoencoder}{{10}{10}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {Autoencoder}{10}}
+\@writefile{nav}{\headcommand {\slideentry {0}{0}{10}{10/10}{}{0}}}
+\@writefile{nav}{\headcommand {\beamer@framepages {10}{10}}}
+\newlabel{For next time<1>}{{11}{11}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {For next time<1>}{11}}
+\newlabel{For next time}{{11}{11}{}{Doc-Start}{}}
+\@writefile{snm}{\beamer@slide {For next time}{11}}
+\@writefile{nav}{\headcommand {\slideentry {0}{0}{11}{11/11}{}{0}}}
+\@writefile{nav}{\headcommand {\beamer@framepages {11}{11}}}
+\@writefile{nav}{\headcommand {\beamer@partpages {1}{11}}}
+\@writefile{nav}{\headcommand {\beamer@subsectionpages {1}{11}}}
+\@writefile{nav}{\headcommand {\beamer@sectionpages {1}{11}}}
+\@writefile{nav}{\headcommand {\beamer@documentpages {11}}}
+\@writefile{nav}{\headcommand {\gdef \inserttotalframenumber {11}}}
+\gdef \@abspage@last{11}

out/main.nav

@@ -0,0 +1,27 @@
+\headcommand {\slideentry {0}{0}{1}{1/1}{}{0}}
+\headcommand {\beamer@framepages {1}{1}}
+\headcommand {\slideentry {0}{0}{2}{2/2}{}{0}}
+\headcommand {\beamer@framepages {2}{2}}
+\headcommand {\slideentry {0}{0}{3}{3/3}{}{0}}
+\headcommand {\beamer@framepages {3}{3}}
+\headcommand {\slideentry {0}{0}{4}{4/4}{}{0}}
+\headcommand {\beamer@framepages {4}{4}}
+\headcommand {\slideentry {0}{0}{5}{5/5}{}{0}}
+\headcommand {\beamer@framepages {5}{5}}
+\headcommand {\slideentry {0}{0}{6}{6/6}{}{0}}
+\headcommand {\beamer@framepages {6}{6}}
+\headcommand {\slideentry {0}{0}{7}{7/7}{}{0}}
+\headcommand {\beamer@framepages {7}{7}}
+\headcommand {\slideentry {0}{0}{8}{8/8}{}{0}}
+\headcommand {\beamer@framepages {8}{8}}
+\headcommand {\slideentry {0}{0}{9}{9/9}{}{0}}
+\headcommand {\beamer@framepages {9}{9}}
+\headcommand {\slideentry {0}{0}{10}{10/10}{}{0}}
+\headcommand {\beamer@framepages {10}{10}}
+\headcommand {\slideentry {0}{0}{11}{11/11}{}{0}}
+\headcommand {\beamer@framepages {11}{11}}
+\headcommand {\beamer@partpages {1}{11}}
+\headcommand {\beamer@subsectionpages {1}{11}}
+\headcommand {\beamer@sectionpages {1}{11}}
+\headcommand {\beamer@documentpages {11}}
+\headcommand {\gdef \inserttotalframenumber {11}}

out/main.snm

@@ -0,0 +1,23 @@
+\beamer@slide {reminder<1>}{2}
+\beamer@slide {reminder}{2}
+\beamer@slide {Big Picture<1>}{3}
+\beamer@slide {Big Picture}{3}
+\beamer@slide {Evaluating your hyperparameter<1>}{4}
+\beamer@slide {Evaluating your hyperparameter}{4}
+\beamer@slide {How to continue<1>}{5}
+\beamer@slide {How to continue}{5}
+\beamer@slide {Intro to Deep Learning<1>}{6}
+\beamer@slide {Intro to Deep Learning}{6}
+\beamer@slide {fig:prep05Intro_to_Deep_Learningadsasdapng}{6}
+\beamer@slide {Intro to Deep Learning<1>}{7}
+\beamer@slide {Intro to Deep Learning}{7}
+\beamer@slide {fig:prep06Intro_to_Deep_Learningadsasdpng}{7}
+\beamer@slide {Intro to Deep Learning<1>}{8}
+\beamer@slide {Intro to Deep Learning}{8}
+\beamer@slide {Autoencoder<1>}{9}
+\beamer@slide {Autoencoder}{9}
+\beamer@slide {fig:prep08Autoencoderaepng}{9}
+\beamer@slide {Autoencoder<1>}{10}
+\beamer@slide {Autoencoder}{10}
+\beamer@slide {For next time<1>}{11}
+\beamer@slide {For next time}{11}

out/main.tex

@@ -0,0 +1,424 @@
+\UseRawInputEncoding
+%\documentclass[hyperref={pdfpagelabels=false}]{beamer}
+\documentclass[hyperref={pdfpagelabels=false},aspectratio=169]{beamer}
+% Die Hyperref Option hyperref={pdfpagelabels=false} verhindert die Warnung:
+% Package hyperref Warning: Option `pdfpagelabels' is turned off
+% (hyperref)                because \thepage is undefined. 
+% Hyperref stopped early 
+%
+
+\usepackage{lmodern}
+% Das Paket lmodern erspart die folgenden Warnungen:
+% LaTeX Font Warning: Font shape `OT1/cmss/m/n' in size <4> not available
+% (Font)              size <5> substituted on input line 22.
+% LaTeX Font Warning: Size substitutions with differences
+% (Font)              up to 1.0pt have occurred.
+%
+
+% Wenn \titel{\ldots} \author{\ldots} erst nach \begin{document} kommen,
+% kommt folgende Warnung:
+% Package hyperref Warning: Option `pdfauthor' has already been used,
+% (hyperref) ... 
+% Daher steht es hier vor \begin{document}
+
+\title[Anomaly Detection and AutoML]{Lets go deeper!}   
+\author{Simon Kluettermann} 
+\date{\today} 
+
+
+ \institute{ls9 tu Dortmund}
+
+
+% Dadurch wird verhindert, dass die Navigationsleiste angezeigt wird.
+\setbeamertemplate{navigation symbols}{}
+
+% zusaetzlich ist das usepackage{beamerthemeshadow} eingebunden 
+\usepackage{beamerthemeshadow}
+
+\hypersetup{pdfstartview={Fit}} % fits the presentation to the window when first displayed
+
+\usepackage{appendixnumberbeamer}
+\usepackage{listings}
+
+
+\usetheme{CambridgeUS}
+\usepackage{ngerman}
+\usecolortheme{dolphin}
+
+
+%  \beamersetuncovermixins{\opaqueness<1>{25}}{\opaqueness<2$\Rightarrow${15}}
+%  sorgt dafuer das die Elemente die erst noch (zukuenftig) kommen 
+%  nur schwach angedeutet erscheinen 
+%\beamersetuncovermixins{\opaqueness<1>{25}}{\opaqueness<2$\Rightarrow${15}}%here disabled
+% klappt auch bei Tabellen, wenn teTeX verwendet wird\ldots
+\renewcommand{\figurename}{}
+
+\setbeamertemplate{footline}
+{
+  \leavevmode%
+  \hbox{%
+  \begin{beamercolorbox}[wd=.4\paperwidth,ht=2.25ex,dp=1ex,center]{author in head/foot}%
+    \usebeamerfont{author in head/foot}\insertshorttitle
+  \end{beamercolorbox}%
+  \begin{beamercolorbox}[wd=.25\paperwidth,ht=2.25ex,dp=1ex,center]{title in head/foot}%
+    \usebeamerfont{title in head/foot}\insertsection
+  \end{beamercolorbox}%
+  \begin{beamercolorbox}[wd=.3499\paperwidth,ht=2.25ex,dp=1ex,right]{date in head/foot}%
+    \usebeamerfont{date in head/foot}\insertshortdate{}\hspace*{2em}
+    \hyperlink{toc}{\insertframenumber{} / \inserttotalframenumber\hspace*{2ex}} 
+  \end{beamercolorbox}}%
+  \vskip0pt%
+}
+
+\usepackage[absolute,overlay]{textpos}
+\usepackage{graphicx}
+
+\newcommand{\source}[1]{\begin{textblock*}{9cm}(0.1cm,8.9cm)
+    \begin{beamercolorbox}[ht=0.5cm,left]{framesource}
+        \usebeamerfont{framesource}\usebeamercolor[fg!66]{framesource} Source: {#1}
+    \end{beamercolorbox}
+\end{textblock*}}
+
+
+\begin{document}
+
+
+
+%from file ../case3/data/000.txt
+\begin{frame}[label=]
+\frametitle{}
+\begin{titlepage}
+
+	\centering
+	{\huge\bfseries \par}
+	\vspace{2cm}
+	{\LARGE\itshape Simon Kluettermann\par}
+	\vspace{1.5cm}
+	{\scshape\Large Master Thesis in Physics\par}
+	\vspace{0.2cm}
+	{\Large submitted to the \par}
+	\vspace{0.2cm}
+	{\scshape\Large Faculty of Mathematics Computer Science and Natural Sciences \par}
+	\vspace{0.2cm}
+	{\Large \par}
+	\vspace{0.2cm}
+	{\scshape\Large RWTH Aachen University}
+	\vspace{1cm}
+	
+	\vfill
+	{\scshape\Large Department of Physics\par}
+	\vspace{0.2cm}
+	{\scshape\Large Insitute for theoretical Particle Physics and Cosmology\par}
+	\vspace{0.2cm}
+	{ \Large\par}
+	\vspace{0.2cm}
+	{\Large First Referee: Prof. Dr. Michael Kraemer \par}
+	{\Large Second Referee: Prof. Dr. Felix Kahlhoefer}
+
+	\vfill
+
+% Bottom of the page
+	{\large November 2020 \par}
+\end{titlepage}
+\pagenumbering{roman}
+\thispagestyle{empty}
+\null
+\newpage
+\setcounter{page}{1}
+\pagenumbering{arabic}
+\end{frame}
+
+
+%from file ../case3/data/001reminder.txt
+\begin{frame}[label=reminder]
+\frametitle{reminder}
+\begin{itemize}
+
+    \item No Case Study next week
+
+\begin{itemize}
+
+    \item neither Tuesday (29.11) nor Thursday (01.12)
+
+    \item if you need help: just write me an email!
+
+
+\end{itemize}
+    \item In two weeks: Case Study switched
+
+\begin{itemize}
+
+    \item Q+A Tuesday (6.12, 14:00) online only
+
+    \item Case Study Meeting Thursday (08.12, 14:00-16:00), in OH12 Room 3.032
+
+
+\end{itemize}
+
+\end{itemize}
+\end{frame}
+
+
+%from file ../case3/data/002Big Picture.txt
+\begin{frame}[label=Big Picture]
+\frametitle{Big Picture}
+\begin{itemize}
+
+    \item Goal for this case study: Have better hyperparameters than pyod!
+
+    \item So each of you: Gets assigned two algorithms!
+
+\begin{itemize}
+
+    \item One fairly simple before
+
+    \item One more complicated one today
+
+
+\end{itemize}
+    \item Try to find the best possible hyperparameters for your algorithms
+
+\begin{itemize}
+
+    \item Try to be clever (for example: PCA: $n_{components}<n_{features}$. Maybe $\frac{n_{components}}{n_{features}}$ constant?
+
+
+\end{itemize}
+    \item Afterwards
+
+\begin{itemize}
+
+    \item Write down your findings into a simple function (given data, what are my best hyperparameters)
+
+    \item Write down your finding into a report (together, double collumn. 6 Pages per student, plus comparison of algorithms to each other)
+
+    \item One final presentation together in front of my colleagues. About 10min per student.
+
+
+\end{itemize}
+
+\end{itemize}
+\end{frame}
+
+
+%from file ../case3/data/003Evaluating your hyperparameter.txt
+\begin{frame}[label=Evaluating your hyperparameter]
+\frametitle{Evaluating your hyperparameter}
+\begin{itemize}
+
+    \item My suggestion: Compare to normal parameters.
+
+    \item This means you get two lists of AUC scores
+
+    \item Your params: [0.80,0.75,0.73,....,0.95]
+
+    \item Pyod params: [0.82,0.71,0.48,....,0.95]
+
+    \item look at two values
+
+    \item $\sum_i your_i-pyod_i$
+
+\begin{itemize}
+
+    \item Total improvment. If positive, then your parameters help;)
+
+    \item But hard to see if this is significant
+
+
+\end{itemize}
+    \item Fraction of $your_i>pyod_i$
+
+\begin{itemize}
+
+    \item Quantised, so does not care about improving your parameters further
+
+    \item But easy to see if this is significant
+
+\begin{itemize}
+
+    \item 0.5$\Rightarrow$Probably just random
+
+    \item 0.9$\Rightarrow$Probably quite significant
+
+
+\end{itemize}
+
+\end{itemize}
+
+\end{itemize}
+\end{frame}
+
+
+%from file ../case3/data/004How to continue.txt
+\begin{frame}[label=How to continue]
+\frametitle{How to continue}
+\begin{itemize}
+
+    \item See how far you can improve this?
+
+    \item Treat this as a supervised optimisation problem: Given this dataset, find the best hyperparameters
+
+    \item Might be useful to look at more input parameters
+
+    \item Might help to formulate your parameters differently
+
+    \item But be aware of \textbf{overfitting}!
+
+
+\end{itemize}
+\end{frame}
+
+
+%from file ../case3/data/005Intro to Deep Learning.txt
+\begin{frame}[label=Intro to Deep Learning]
+\frametitle{Intro to Deep Learning}
+\begin{figure}[H] 
+  \centering
+\includegraphics[width=0.9\textwidth]{..//prep/05Intro_to_Deep_Learning/adsasda.png}
+\label{fig:prep05Intro_to_Deep_Learningadsasdapng}
+  \end{figure}
+
+
+\end{frame}
+
+
+%from file ../case3/data/006Intro to Deep Learning.txt
+\begin{frame}[label=Intro to Deep Learning]
+\frametitle{Intro to Deep Learning}
+\begin{figure}[H] 
+  \centering
+\includegraphics[width=0.9\textwidth]{..//prep/06Intro_to_Deep_Learning/adsasd.png}
+\label{fig:prep06Intro_to_Deep_Learningadsasdpng}
+  \end{figure}
+
+
+\end{frame}
+
+
+%from file ../case3/data/007Intro to Deep Learning.txt
+\begin{frame}[label=Intro to Deep Learning]
+\frametitle{Intro to Deep Learning}
+\begin{itemize}
+
+    \item The idea is always the same:
+
+\begin{itemize}
+
+    \item Define complicated model to learn (often millions of parameters)
+
+    \item Define loss function that this model should minimize (example: $\sum_i (y_i-f(x_i))^2$)
+
+    \item Find parameters that minimize the loss ($\Rightarrow$Backpropagation)
+
+
+\end{itemize}
+    \item Usually Neural Networks:
+
+\begin{itemize}
+
+    \item $f(x)=f_n(x)=activation(A_n\cdot f_{n-1}(x)+b_n)$
+
+    \item $f_0(x)=x$
+
+
+\end{itemize}
+    \item Powerful, as you can show that when there are 3 Layers+ (and infinitely sized matrices), you can approximate any function
+
+    \item $\Rightarrow$So a model becomes a loss function
+
+
+\end{itemize}
+\end{frame}
+
+
+%from file ../case3/data/008Autoencoder.txt
+\begin{frame}[label=Autoencoder]
+\frametitle{Autoencoder}
+\begin{figure}[H] 
+  \centering
+\includegraphics[width=0.9\textwidth]{..//prep/08Autoencoder/ae.png}
+\label{fig:prep08Autoencoderaepng}
+  \end{figure}
+
+
+\end{frame}
+
+
+%from file ../case3/data/009Autoencoder.txt
+\begin{frame}[label=Autoencoder]
+\frametitle{Autoencoder}
+\begin{itemize}
+
+    \item Lets look at some of its Hyperparameters
+
+    \item Autoencoder Specific
+
+\begin{itemize}
+
+    \item Compression factor (Latent space size)
+
+    \item Loss function (mse?)
+
+
+\end{itemize}
+    \item Neural Network architecture
+
+\begin{itemize}
+
+    \item Number of layers
+
+    \item Number of neurons in each layer (Shape of the matrices $A_n$)
+
+
+\end{itemize}
+    \item Optimisation parameters
+
+\begin{itemize}
+
+    \item Learning Rate
+
+\begin{itemize}
+
+    \item Controls how fast the parameters are found
+
+    \item To high value makes the training unstable
+
+
+\end{itemize}
+    \item Batch size
+
+\begin{itemize}
+
+    \item Controls how many samples are averaged together.
+
+    \item Lower values make the training more stable, but also the result less optimal
+
+
+\end{itemize}
+
+\end{itemize}
+
+\end{itemize}
+\end{frame}
+
+
+%from file ../case3/data/010For next time.txt
+\begin{frame}[label=For next time]
+\frametitle{For next time}
+\begin{itemize}
+
+    \item (if you have not finished finding good parameters for your old algorithm, continue searching for them)
+
+    \item Take a look at your new algorithm
+
+    \item Run it once on cardio, take a look at which parameters you have
+
+    \item Prepare a similar presentation to last time (include your cardio result)
+
+
+\end{itemize}
+\end{frame}
+
+
+
+\end{document}

prep/000/q

@@ -0,0 +1 @@
+<titlepage>

prep/01reminder/q

@@ -0,0 +1,11 @@
+No Case Study next week
+<l2st>
+neither Tuesday (29.11) nor Thursday (01.12)
+if you need help: just write me an email!
+</l2st>
+In two weeks: Case Study switched
+<l2st>
+Q+A Tuesday (6.12, 14:00) online only
+Case Study Meeting Thursday (08.12, 14:00-16:00), in OH12 Room 3.032
+</l2st>
+

prep/02Big_Picture/q

@@ -0,0 +1,16 @@
+Goal for this case study: Have better hyperparameters than pyod!
+So each of you: Gets assigned two algorithms!
+<l2st>
+One fairly simple before
+One more complicated one today
+</l2st>
+Try to find the best possible hyperparameters for your algorithms
+<l2st>
+Try to be clever (for example: PCA: $n_{components}<n_{features}$. Maybe $\frac{n_{components}}{n_{features}}$ constant?
+</l2st>
+Afterwards
+<l2st>
+Write down your findings into a simple function (given data, what are my best hyperparameters)
+Write down your finding into a report (together, double collumn. 6 Pages per student, plus comparison of algorithms to each other)
+One final presentation together in front of my colleagues. About 10min per student.
+</l2st>

prep/03Evaluating_your_hyperparameter/q

@@ -0,0 +1,21 @@
+My suggestion: Compare to normal parameters.
+This means you get two lists of AUC scores
+Your params: [0.80,0.75,0.73,....,0.95]
+Pyod params: [0.82,0.71,0.48,....,0.95]
+look at two values
+$\sum_i your_i-pyod_i$
+<l2st>
+Total improvment. If positive, then your parameters help;)
+But hard to see if this is significant    
+</l2st>
+Fraction of $your_i>pyod_i$
+<l2st>
+Quantised, so does not care about improving your parameters further
+But easy to see if this is significant
+<l3st>
+0.5->Probably just random
+0.9->Probably quite significant
+</l3st>
+</l2st>
+
+

prep/04How_to_continue/q

@@ -0,0 +1,5 @@
+See how far you can improve this?
+Treat this as a supervised optimisation problem: Given this dataset, find the best hyperparameters
+Might be useful to look at more input parameters
+Might help to formulate your parameters differently
+But be aware of \textbf{overfitting}!

prep/07Intro_to_Deep_Learning/q

@@ -0,0 +1,16 @@
+The idea is always the same:
+<l2st>
+Define complicated model to learn (often millions of parameters)
+Define loss function that this model should minimize (example: $\sum_i (y_i-f(x_i))^2$)
+Find parameters that minimize the loss (->Backpropagation)
+</l2st>
+Usually Neural Networks:
+<l2st>
+$f(x)=f_n(x)=activation(A_n\cdot f_{n-1}(x)+b_n)$
+$f_0(x)=x$
+</l2st>
+Powerful, as you can show that when there are 3 Layers+ (and infinitely sized matrices), you can approximate any function
+->So a model becomes a loss function
+
+
+

prep/09Autoencoder/q

@@ -0,0 +1,24 @@
+Lets look at some of its Hyperparameters
+Autoencoder Specific
+<l2st>
+Compression factor (Latent space size)
+Loss function (mse?)
+</l2st>
+Neural Network architecture
+<l2st>
+Number of layers
+Number of neurons in each layer (Shape of the matrices $A_n$)
+</l2st>
+Optimisation parameters
+<l2st>
+Learning Rate
+<l3st>
+Controls how fast the parameters are found
+To high value makes the training unstable
+</l3st>
+Batch size
+<l3st>
+Controls how many samples are averaged together.
+Lower values make the training more stable, but also the result less optimal
+</l3st>
+</l2st>

prep/10For_next_time/q

@@ -0,0 +1,4 @@
+(if you have not finished finding good parameters for your old algorithm, continue searching for them)
+Take a look at your new algorithm
+Run it once on cardio, take a look at which parameters you have
+Prepare a similar presentation to last time (include your cardio result)