initial push
|
@ -0,0 +1,5 @@
|
||||||
|
<frame >
|
||||||
|
|
||||||
|
<titlepage>
|
||||||
|
|
||||||
|
</frame>
|
|
@ -0,0 +1,14 @@
|
||||||
|
<frame title="reminder">
|
||||||
|
<list>
|
||||||
|
<e>No Case Study next week</e>
|
||||||
|
<l2st>
|
||||||
|
<e>neither Tuesday (29.11) nor Thursday (01.12)</e>
|
||||||
|
<e>if you need help: just write me an email!</e>
|
||||||
|
</l2st>
|
||||||
|
<e>In two weeks: Case Study switched</e>
|
||||||
|
<l2st>
|
||||||
|
<e>Q+A Tuesday (6.12, 14:00) online only</e>
|
||||||
|
<e>Case Study Meeting Thursday (08.12, 14:00-16:00), in OH12 Room 3.032</e>
|
||||||
|
</l2st>
|
||||||
|
</list>
|
||||||
|
</frame>
|
|
@ -0,0 +1,20 @@
|
||||||
|
<frame title="Big Picture">
|
||||||
|
<list>
|
||||||
|
<e>Goal for this case study: Have better hyperparameters than pyod!</e>
|
||||||
|
<e>So each of you: Gets assigned two algorithms!</e>
|
||||||
|
<l2st>
|
||||||
|
<e>One fairly simple before</e>
|
||||||
|
<e>One more complicated one today</e>
|
||||||
|
</l2st>
|
||||||
|
<e>Try to find the best possible hyperparameters for your algorithms</e>
|
||||||
|
<l2st>
|
||||||
|
<e>Try to be clever (for example: PCA: $n_{components}<n_{features}$. Maybe $\frac{n_{components}}{n_{features}}$ constant?</e>
|
||||||
|
</l2st>
|
||||||
|
<e>Afterwards</e>
|
||||||
|
<l2st>
|
||||||
|
<e>Write down your findings into a simple function (given data, what are my best hyperparameters)</e>
|
||||||
|
<e>Write down your finding into a report (together, double collumn. 6 Pages per student, plus comparison of algorithms to each other)</e>
|
||||||
|
<e>One final presentation together in front of my colleagues. About 10min per student.</e>
|
||||||
|
</l2st>
|
||||||
|
</list>
|
||||||
|
</frame>
|
|
@ -0,0 +1,23 @@
|
||||||
|
<frame title="Evaluating your hyperparameter">
|
||||||
|
<list>
|
||||||
|
<e>My suggestion: Compare to normal parameters.</e>
|
||||||
|
<e>This means you get two lists of AUC scores</e>
|
||||||
|
<e>Your params: [0.80,0.75,0.73,....,0.95]</e>
|
||||||
|
<e>Pyod params: [0.82,0.71,0.48,....,0.95]</e>
|
||||||
|
<e>look at two values</e>
|
||||||
|
<e>$\sum_i your_i-pyod_i$</e>
|
||||||
|
<l2st>
|
||||||
|
<e>Total improvment. If positive, then your parameters help;)</e>
|
||||||
|
<e>But hard to see if this is significant </e>
|
||||||
|
</l2st>
|
||||||
|
<e>Fraction of $your_i>pyod_i$</e>
|
||||||
|
<l2st>
|
||||||
|
<e>Quantised, so does not care about improving your parameters further</e>
|
||||||
|
<e>But easy to see if this is significant</e>
|
||||||
|
<l3st>
|
||||||
|
<e>0.5->Probably just random</e>
|
||||||
|
<e>0.9->Probably quite significant</e>
|
||||||
|
</l3st>
|
||||||
|
</l2st>
|
||||||
|
</list>
|
||||||
|
</frame>
|
|
@ -0,0 +1,9 @@
|
||||||
|
<frame title="How to continue">
|
||||||
|
<list>
|
||||||
|
<e>See how far you can improve this?</e>
|
||||||
|
<e>Treat this as a supervised optimisation problem: Given this dataset, find the best hyperparameters</e>
|
||||||
|
<e>Might be useful to look at more input parameters</e>
|
||||||
|
<e>Might help to formulate your parameters differently</e>
|
||||||
|
<e>But be aware of \textbf{overfitting}!</e>
|
||||||
|
</list>
|
||||||
|
</frame>
|
|
@ -0,0 +1,3 @@
|
||||||
|
<frame title="Intro to Deep Learning">
|
||||||
|
<i f="..//prep/05Intro_to_Deep_Learning/adsasda.png" wmode="True"></i>
|
||||||
|
</frame>
|
|
@ -0,0 +1,3 @@
|
||||||
|
<frame title="Intro to Deep Learning">
|
||||||
|
<i f="..//prep/06Intro_to_Deep_Learning/adsasd.png" wmode="True"></i>
|
||||||
|
</frame>
|
|
@ -0,0 +1,17 @@
|
||||||
|
<frame title="Intro to Deep Learning">
|
||||||
|
<list>
|
||||||
|
<e>The idea is always the same:</e>
|
||||||
|
<l2st>
|
||||||
|
<e>Define complicated model to learn (often millions of parameters)</e>
|
||||||
|
<e>Define loss function that this model should minimize (example: $\sum_i (y_i-f(x_i))^2$)</e>
|
||||||
|
<e>Find parameters that minimize the loss (->Backpropagation)</e>
|
||||||
|
</l2st>
|
||||||
|
<e>Usually Neural Networks:</e>
|
||||||
|
<l2st>
|
||||||
|
<e>$f(x)=f_n(x)=activation(A_n\cdot f_{n-1}(x)+b_n)$</e>
|
||||||
|
<e>$f_0(x)=x$</e>
|
||||||
|
</l2st>
|
||||||
|
<e>Powerful, as you can show that when there are 3 Layers+ (and infinitely sized matrices), you can approximate any function</e>
|
||||||
|
<e>->So a model becomes a loss function</e>
|
||||||
|
</list>
|
||||||
|
</frame>
|
|
@ -0,0 +1,3 @@
|
||||||
|
<frame title="Autoencoder">
|
||||||
|
<i f="..//prep/08Autoencoder/ae.png" wmode="True"></i>
|
||||||
|
</frame>
|
|
@ -0,0 +1,28 @@
|
||||||
|
<frame title="Autoencoder">
|
||||||
|
<list>
|
||||||
|
<e>Lets look at some of its Hyperparameters</e>
|
||||||
|
<e>Autoencoder Specific</e>
|
||||||
|
<l2st>
|
||||||
|
<e>Compression factor (Latent space size)</e>
|
||||||
|
<e>Loss function (mse?)</e>
|
||||||
|
</l2st>
|
||||||
|
<e>Neural Network architecture</e>
|
||||||
|
<l2st>
|
||||||
|
<e>Number of layers</e>
|
||||||
|
<e>Number of neurons in each layer (Shape of the matrices $A_n$)</e>
|
||||||
|
</l2st>
|
||||||
|
<e>Optimisation parameters</e>
|
||||||
|
<l2st>
|
||||||
|
<e>Learning Rate</e>
|
||||||
|
<l3st>
|
||||||
|
<e>Controls how fast the parameters are found</e>
|
||||||
|
<e>To high value makes the training unstable</e>
|
||||||
|
</l3st>
|
||||||
|
<e>Batch size</e>
|
||||||
|
<l3st>
|
||||||
|
<e>Controls how many samples are averaged together.</e>
|
||||||
|
<e>Lower values make the training more stable, but also the result less optimal</e>
|
||||||
|
</l3st>
|
||||||
|
</l2st>
|
||||||
|
</list>
|
||||||
|
</frame>
|
|
@ -0,0 +1,8 @@
|
||||||
|
<frame title="For next time">
|
||||||
|
<list>
|
||||||
|
<e>(if you have not finished finding good parameters for your old algorithm, continue searching for them)</e>
|
||||||
|
<e>Take a look at your new algorithm</e>
|
||||||
|
<e>Run it once on cardio, take a look at which parameters you have</e>
|
||||||
|
<e>Prepare a similar presentation to last time (include your cardio result)</e>
|
||||||
|
</list>
|
||||||
|
</frame>
|
|
@ -0,0 +1,12 @@
|
||||||
|
<plt>
|
||||||
|
|
||||||
|
<name Current experiment status>
|
||||||
|
<title Lets go deeper!>
|
||||||
|
<stitle Anomaly Detection and AutoML>
|
||||||
|
|
||||||
|
<institute ls9 tu Dortmund>
|
||||||
|
|
||||||
|
<theme CambridgeUS>
|
||||||
|
<colo dolphin>
|
||||||
|
|
||||||
|
</plt>
|
|
@ -0,0 +1,6 @@
|
||||||
|
Two distributions
|
||||||
|
<l2st>
|
||||||
|
One known (=normal)
|
||||||
|
One unknown (=anomalies)
|
||||||
|
</l2st>
|
||||||
|
Seperate them
|
|
@ -0,0 +1,7 @@
|
||||||
|
Two distributions
|
||||||
|
<l2st>
|
||||||
|
One known (=normal)
|
||||||
|
One unknown (=anomalies)
|
||||||
|
</l2st>
|
||||||
|
Seperate them
|
||||||
|
Problem: few anomalies
|
|
@ -0,0 +1,8 @@
|
||||||
|
Anomalies are rare, so often only a few datapoints known (e.g. Machine Failure in an Aircraft)
|
||||||
|
In practice, anomalies might appear that are not known during testing
|
||||||
|
->So train the model only on normal samples
|
||||||
|
Unsupervised Machine Learning
|
||||||
|
<l2st>
|
||||||
|
What can we say without knowing anomalies?
|
||||||
|
''Understand you dataset''
|
||||||
|
</l2st>
|
|
@ -0,0 +1,8 @@
|
||||||
|
Anomalies are rare, so often only a few datapoints known (e.g. Machine Failure in an Aircraft)
|
||||||
|
In practice, anomalies might appear that are not known during testing
|
||||||
|
->So train the model only on normal samples
|
||||||
|
Unsupervised Machine Learning
|
||||||
|
<l2st>
|
||||||
|
What can we say without knowing anomalies?
|
||||||
|
''Understand you dataset''
|
||||||
|
</l2st>
|
After Width: | Height: | Size: 648 KiB |
|
@ -0,0 +1,6 @@
|
||||||
|
Seems easy? Now do this
|
||||||
|
<l2st>
|
||||||
|
in thousands of dimensions
|
||||||
|
with complicated distributions
|
||||||
|
and overlap between anomalies and normal points
|
||||||
|
</l2st>
|
After Width: | Height: | Size: 47 KiB |
|
@ -0,0 +1,3 @@
|
||||||
|
Most machine learning requires Hyperparameter Optimisation
|
||||||
|
(Find model parameters that result in the best results)
|
||||||
|
->AutoML: Do this automatically as fast as possible
|
|
@ -0,0 +1,9 @@
|
||||||
|
So lets combine both (Auto Anomaly Detection)
|
||||||
|
->Problem
|
||||||
|
<l2st>
|
||||||
|
AutoMl requires Evaluation (loss, accuracy, AUC) to optimize
|
||||||
|
AD can only be evaluated with regards to the anomalies
|
||||||
|
->no longer unsupervised
|
||||||
|
</l2st>
|
||||||
|
So most Anomaly Detection is ''unoptimized''
|
||||||
|
|
|
@ -0,0 +1,3 @@
|
||||||
|
So how to solve this?
|
||||||
|
One option: Think of some function to evaluate only the normal points
|
||||||
|
->A bit hard to do in a case study
|
|
@ -0,0 +1,5 @@
|
||||||
|
So how to solve this?
|
||||||
|
One option: ''Just find the best solution directly''
|
||||||
|
->Zero Shot AutoML
|
||||||
|
Find best practices for hyperparameters
|
||||||
|
Requires optimisation for each model seperately -> matches the case study structure quite well!
|
|
@ -0,0 +1,12 @@
|
||||||
|
Basics of Scientific Computing
|
||||||
|
Basics of AD
|
||||||
|
Basics of AutoML
|
||||||
|
Build groups for each algorithm
|
||||||
|
<l2st>
|
||||||
|
Choose a set of Hyperparameters
|
||||||
|
Find ''best practice`s'' for them
|
||||||
|
Maybe consider more complicated Transformations (Preprocessing, Ensemble)
|
||||||
|
</l2st>
|
||||||
|
Compare between groups (best algorithm for current situation)
|
||||||
|
Evaluate on new datasets
|
||||||
|
Write a report/Present your work
|
After Width: | Height: | Size: 80 KiB |
|
@ -0,0 +1,7 @@
|
||||||
|
Requirements:
|
||||||
|
<l2st>
|
||||||
|
MD Req 1->MD Req 8
|
||||||
|
Basic Python/Math Knowledge
|
||||||
|
Motivation to learn something new;)
|
||||||
|
</l2st>
|
||||||
|
Registration till Saturday, by Email to Simon.Kluettermann@cs.tu-dortmund.de
|
|
@ -0,0 +1,3 @@
|
||||||
|
pdflatex main.tex
|
||||||
|
pdflatex main.tex
|
||||||
|
|
|
@ -0,0 +1,3 @@
|
||||||
|
pdflatex main.tex
|
||||||
|
pdflatex main.tex
|
||||||
|
|
|
@ -0,0 +1,29 @@
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"typ": "img",
|
||||||
|
"files": [
|
||||||
|
"..//prep/05Intro_to_Deep_Learning/adsasda.png"
|
||||||
|
],
|
||||||
|
"label": "prep05Intro_to_Deep_Learningadsasdapng",
|
||||||
|
"caption": "",
|
||||||
|
"where": "../case3/data/005Intro to Deep Learning.txt"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"typ": "img",
|
||||||
|
"files": [
|
||||||
|
"..//prep/06Intro_to_Deep_Learning/adsasd.png"
|
||||||
|
],
|
||||||
|
"label": "prep06Intro_to_Deep_Learningadsasdpng",
|
||||||
|
"caption": "",
|
||||||
|
"where": "../case3/data/006Intro to Deep Learning.txt"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"typ": "img",
|
||||||
|
"files": [
|
||||||
|
"..//prep/08Autoencoder/ae.png"
|
||||||
|
],
|
||||||
|
"label": "prep08Autoencoderaepng",
|
||||||
|
"caption": "",
|
||||||
|
"where": "../case3/data/008Autoencoder.txt"
|
||||||
|
}
|
||||||
|
]
|
|
@ -0,0 +1,92 @@
|
||||||
|
\relax
|
||||||
|
\providecommand\hyper@newdestlabel[2]{}
|
||||||
|
\providecommand\HyperFirstAtBeginDocument{\AtBeginDocument}
|
||||||
|
\HyperFirstAtBeginDocument{\ifx\hyper@anchor\@undefined
|
||||||
|
\global\let\oldcontentsline\contentsline
|
||||||
|
\gdef\contentsline#1#2#3#4{\oldcontentsline{#1}{#2}{#3}}
|
||||||
|
\global\let\oldnewlabel\newlabel
|
||||||
|
\gdef\newlabel#1#2{\newlabelxx{#1}#2}
|
||||||
|
\gdef\newlabelxx#1#2#3#4#5#6{\oldnewlabel{#1}{{#2}{#3}}}
|
||||||
|
\AtEndDocument{\ifx\hyper@anchor\@undefined
|
||||||
|
\let\contentsline\oldcontentsline
|
||||||
|
\let\newlabel\oldnewlabel
|
||||||
|
\fi}
|
||||||
|
\fi}
|
||||||
|
\global\let\hyper@last\relax
|
||||||
|
\gdef\HyperFirstAtBeginDocument#1{#1}
|
||||||
|
\providecommand\HyField@AuxAddToFields[1]{}
|
||||||
|
\providecommand\HyField@AuxAddToCoFields[2]{}
|
||||||
|
\@writefile{nav}{\headcommand {\slideentry {0}{0}{1}{1/1}{}{0}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@framepages {1}{1}}}
|
||||||
|
\newlabel{reminder<1>}{{2}{2}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {reminder<1>}{2}}
|
||||||
|
\newlabel{reminder}{{2}{2}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {reminder}{2}}
|
||||||
|
\@writefile{nav}{\headcommand {\slideentry {0}{0}{2}{2/2}{}{0}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@framepages {2}{2}}}
|
||||||
|
\newlabel{Big Picture<1>}{{3}{3}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Big Picture<1>}{3}}
|
||||||
|
\newlabel{Big Picture}{{3}{3}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Big Picture}{3}}
|
||||||
|
\@writefile{nav}{\headcommand {\slideentry {0}{0}{3}{3/3}{}{0}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@framepages {3}{3}}}
|
||||||
|
\newlabel{Evaluating your hyperparameter<1>}{{4}{4}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Evaluating your hyperparameter<1>}{4}}
|
||||||
|
\newlabel{Evaluating your hyperparameter}{{4}{4}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Evaluating your hyperparameter}{4}}
|
||||||
|
\@writefile{nav}{\headcommand {\slideentry {0}{0}{4}{4/4}{}{0}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@framepages {4}{4}}}
|
||||||
|
\newlabel{How to continue<1>}{{5}{5}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {How to continue<1>}{5}}
|
||||||
|
\newlabel{How to continue}{{5}{5}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {How to continue}{5}}
|
||||||
|
\@writefile{nav}{\headcommand {\slideentry {0}{0}{5}{5/5}{}{0}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@framepages {5}{5}}}
|
||||||
|
\newlabel{Intro to Deep Learning<1>}{{6}{6}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Intro to Deep Learning<1>}{6}}
|
||||||
|
\newlabel{Intro to Deep Learning}{{6}{6}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Intro to Deep Learning}{6}}
|
||||||
|
\newlabel{fig:prep05Intro_to_Deep_Learningadsasdapng}{{6}{6}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {fig:prep05Intro_to_Deep_Learningadsasdapng}{6}}
|
||||||
|
\@writefile{nav}{\headcommand {\slideentry {0}{0}{6}{6/6}{}{0}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@framepages {6}{6}}}
|
||||||
|
\newlabel{Intro to Deep Learning<1>}{{7}{7}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Intro to Deep Learning<1>}{7}}
|
||||||
|
\newlabel{Intro to Deep Learning}{{7}{7}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Intro to Deep Learning}{7}}
|
||||||
|
\newlabel{fig:prep06Intro_to_Deep_Learningadsasdpng}{{7}{7}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {fig:prep06Intro_to_Deep_Learningadsasdpng}{7}}
|
||||||
|
\@writefile{nav}{\headcommand {\slideentry {0}{0}{7}{7/7}{}{0}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@framepages {7}{7}}}
|
||||||
|
\newlabel{Intro to Deep Learning<1>}{{8}{8}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Intro to Deep Learning<1>}{8}}
|
||||||
|
\newlabel{Intro to Deep Learning}{{8}{8}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Intro to Deep Learning}{8}}
|
||||||
|
\@writefile{nav}{\headcommand {\slideentry {0}{0}{8}{8/8}{}{0}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@framepages {8}{8}}}
|
||||||
|
\newlabel{Autoencoder<1>}{{9}{9}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Autoencoder<1>}{9}}
|
||||||
|
\newlabel{Autoencoder}{{9}{9}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Autoencoder}{9}}
|
||||||
|
\newlabel{fig:prep08Autoencoderaepng}{{9}{9}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {fig:prep08Autoencoderaepng}{9}}
|
||||||
|
\@writefile{nav}{\headcommand {\slideentry {0}{0}{9}{9/9}{}{0}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@framepages {9}{9}}}
|
||||||
|
\newlabel{Autoencoder<1>}{{10}{10}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Autoencoder<1>}{10}}
|
||||||
|
\newlabel{Autoencoder}{{10}{10}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {Autoencoder}{10}}
|
||||||
|
\@writefile{nav}{\headcommand {\slideentry {0}{0}{10}{10/10}{}{0}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@framepages {10}{10}}}
|
||||||
|
\newlabel{For next time<1>}{{11}{11}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {For next time<1>}{11}}
|
||||||
|
\newlabel{For next time}{{11}{11}{}{Doc-Start}{}}
|
||||||
|
\@writefile{snm}{\beamer@slide {For next time}{11}}
|
||||||
|
\@writefile{nav}{\headcommand {\slideentry {0}{0}{11}{11/11}{}{0}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@framepages {11}{11}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@partpages {1}{11}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@subsectionpages {1}{11}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@sectionpages {1}{11}}}
|
||||||
|
\@writefile{nav}{\headcommand {\beamer@documentpages {11}}}
|
||||||
|
\@writefile{nav}{\headcommand {\gdef \inserttotalframenumber {11}}}
|
||||||
|
\gdef \@abspage@last{11}
|
|
@ -0,0 +1,27 @@
|
||||||
|
\headcommand {\slideentry {0}{0}{1}{1/1}{}{0}}
|
||||||
|
\headcommand {\beamer@framepages {1}{1}}
|
||||||
|
\headcommand {\slideentry {0}{0}{2}{2/2}{}{0}}
|
||||||
|
\headcommand {\beamer@framepages {2}{2}}
|
||||||
|
\headcommand {\slideentry {0}{0}{3}{3/3}{}{0}}
|
||||||
|
\headcommand {\beamer@framepages {3}{3}}
|
||||||
|
\headcommand {\slideentry {0}{0}{4}{4/4}{}{0}}
|
||||||
|
\headcommand {\beamer@framepages {4}{4}}
|
||||||
|
\headcommand {\slideentry {0}{0}{5}{5/5}{}{0}}
|
||||||
|
\headcommand {\beamer@framepages {5}{5}}
|
||||||
|
\headcommand {\slideentry {0}{0}{6}{6/6}{}{0}}
|
||||||
|
\headcommand {\beamer@framepages {6}{6}}
|
||||||
|
\headcommand {\slideentry {0}{0}{7}{7/7}{}{0}}
|
||||||
|
\headcommand {\beamer@framepages {7}{7}}
|
||||||
|
\headcommand {\slideentry {0}{0}{8}{8/8}{}{0}}
|
||||||
|
\headcommand {\beamer@framepages {8}{8}}
|
||||||
|
\headcommand {\slideentry {0}{0}{9}{9/9}{}{0}}
|
||||||
|
\headcommand {\beamer@framepages {9}{9}}
|
||||||
|
\headcommand {\slideentry {0}{0}{10}{10/10}{}{0}}
|
||||||
|
\headcommand {\beamer@framepages {10}{10}}
|
||||||
|
\headcommand {\slideentry {0}{0}{11}{11/11}{}{0}}
|
||||||
|
\headcommand {\beamer@framepages {11}{11}}
|
||||||
|
\headcommand {\beamer@partpages {1}{11}}
|
||||||
|
\headcommand {\beamer@subsectionpages {1}{11}}
|
||||||
|
\headcommand {\beamer@sectionpages {1}{11}}
|
||||||
|
\headcommand {\beamer@documentpages {11}}
|
||||||
|
\headcommand {\gdef \inserttotalframenumber {11}}
|
|
@ -0,0 +1,23 @@
|
||||||
|
\beamer@slide {reminder<1>}{2}
|
||||||
|
\beamer@slide {reminder}{2}
|
||||||
|
\beamer@slide {Big Picture<1>}{3}
|
||||||
|
\beamer@slide {Big Picture}{3}
|
||||||
|
\beamer@slide {Evaluating your hyperparameter<1>}{4}
|
||||||
|
\beamer@slide {Evaluating your hyperparameter}{4}
|
||||||
|
\beamer@slide {How to continue<1>}{5}
|
||||||
|
\beamer@slide {How to continue}{5}
|
||||||
|
\beamer@slide {Intro to Deep Learning<1>}{6}
|
||||||
|
\beamer@slide {Intro to Deep Learning}{6}
|
||||||
|
\beamer@slide {fig:prep05Intro_to_Deep_Learningadsasdapng}{6}
|
||||||
|
\beamer@slide {Intro to Deep Learning<1>}{7}
|
||||||
|
\beamer@slide {Intro to Deep Learning}{7}
|
||||||
|
\beamer@slide {fig:prep06Intro_to_Deep_Learningadsasdpng}{7}
|
||||||
|
\beamer@slide {Intro to Deep Learning<1>}{8}
|
||||||
|
\beamer@slide {Intro to Deep Learning}{8}
|
||||||
|
\beamer@slide {Autoencoder<1>}{9}
|
||||||
|
\beamer@slide {Autoencoder}{9}
|
||||||
|
\beamer@slide {fig:prep08Autoencoderaepng}{9}
|
||||||
|
\beamer@slide {Autoencoder<1>}{10}
|
||||||
|
\beamer@slide {Autoencoder}{10}
|
||||||
|
\beamer@slide {For next time<1>}{11}
|
||||||
|
\beamer@slide {For next time}{11}
|
|
@ -0,0 +1,424 @@
|
||||||
|
\UseRawInputEncoding
|
||||||
|
%\documentclass[hyperref={pdfpagelabels=false}]{beamer}
|
||||||
|
\documentclass[hyperref={pdfpagelabels=false},aspectratio=169]{beamer}
|
||||||
|
% Die Hyperref Option hyperref={pdfpagelabels=false} verhindert die Warnung:
|
||||||
|
% Package hyperref Warning: Option `pdfpagelabels' is turned off
|
||||||
|
% (hyperref) because \thepage is undefined.
|
||||||
|
% Hyperref stopped early
|
||||||
|
%
|
||||||
|
|
||||||
|
\usepackage{lmodern}
|
||||||
|
% Das Paket lmodern erspart die folgenden Warnungen:
|
||||||
|
% LaTeX Font Warning: Font shape `OT1/cmss/m/n' in size <4> not available
|
||||||
|
% (Font) size <5> substituted on input line 22.
|
||||||
|
% LaTeX Font Warning: Size substitutions with differences
|
||||||
|
% (Font) up to 1.0pt have occurred.
|
||||||
|
%
|
||||||
|
|
||||||
|
% Wenn \titel{\ldots} \author{\ldots} erst nach \begin{document} kommen,
|
||||||
|
% kommt folgende Warnung:
|
||||||
|
% Package hyperref Warning: Option `pdfauthor' has already been used,
|
||||||
|
% (hyperref) ...
|
||||||
|
% Daher steht es hier vor \begin{document}
|
||||||
|
|
||||||
|
\title[Anomaly Detection and AutoML]{Lets go deeper!}
|
||||||
|
\author{Simon Kluettermann}
|
||||||
|
\date{\today}
|
||||||
|
|
||||||
|
|
||||||
|
\institute{ls9 tu Dortmund}
|
||||||
|
|
||||||
|
|
||||||
|
% Dadurch wird verhindert, dass die Navigationsleiste angezeigt wird.
|
||||||
|
\setbeamertemplate{navigation symbols}{}
|
||||||
|
|
||||||
|
% zusaetzlich ist das usepackage{beamerthemeshadow} eingebunden
|
||||||
|
\usepackage{beamerthemeshadow}
|
||||||
|
|
||||||
|
\hypersetup{pdfstartview={Fit}} % fits the presentation to the window when first displayed
|
||||||
|
|
||||||
|
\usepackage{appendixnumberbeamer}
|
||||||
|
\usepackage{listings}
|
||||||
|
|
||||||
|
|
||||||
|
\usetheme{CambridgeUS}
|
||||||
|
\usepackage{ngerman}
|
||||||
|
\usecolortheme{dolphin}
|
||||||
|
|
||||||
|
|
||||||
|
% \beamersetuncovermixins{\opaqueness<1>{25}}{\opaqueness<2$\Rightarrow${15}}
|
||||||
|
% sorgt dafuer das die Elemente die erst noch (zukuenftig) kommen
|
||||||
|
% nur schwach angedeutet erscheinen
|
||||||
|
%\beamersetuncovermixins{\opaqueness<1>{25}}{\opaqueness<2$\Rightarrow${15}}%here disabled
|
||||||
|
% klappt auch bei Tabellen, wenn teTeX verwendet wird\ldots
|
||||||
|
\renewcommand{\figurename}{}
|
||||||
|
|
||||||
|
\setbeamertemplate{footline}
|
||||||
|
{
|
||||||
|
\leavevmode%
|
||||||
|
\hbox{%
|
||||||
|
\begin{beamercolorbox}[wd=.4\paperwidth,ht=2.25ex,dp=1ex,center]{author in head/foot}%
|
||||||
|
\usebeamerfont{author in head/foot}\insertshorttitle
|
||||||
|
\end{beamercolorbox}%
|
||||||
|
\begin{beamercolorbox}[wd=.25\paperwidth,ht=2.25ex,dp=1ex,center]{title in head/foot}%
|
||||||
|
\usebeamerfont{title in head/foot}\insertsection
|
||||||
|
\end{beamercolorbox}%
|
||||||
|
\begin{beamercolorbox}[wd=.3499\paperwidth,ht=2.25ex,dp=1ex,right]{date in head/foot}%
|
||||||
|
\usebeamerfont{date in head/foot}\insertshortdate{}\hspace*{2em}
|
||||||
|
\hyperlink{toc}{\insertframenumber{} / \inserttotalframenumber\hspace*{2ex}}
|
||||||
|
\end{beamercolorbox}}%
|
||||||
|
\vskip0pt%
|
||||||
|
}
|
||||||
|
|
||||||
|
\usepackage[absolute,overlay]{textpos}
|
||||||
|
\usepackage{graphicx}
|
||||||
|
|
||||||
|
\newcommand{\source}[1]{\begin{textblock*}{9cm}(0.1cm,8.9cm)
|
||||||
|
\begin{beamercolorbox}[ht=0.5cm,left]{framesource}
|
||||||
|
\usebeamerfont{framesource}\usebeamercolor[fg!66]{framesource} Source: {#1}
|
||||||
|
\end{beamercolorbox}
|
||||||
|
\end{textblock*}}
|
||||||
|
|
||||||
|
|
||||||
|
\begin{document}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
%from file ../case3/data/000.txt
|
||||||
|
\begin{frame}[label=]
|
||||||
|
\frametitle{}
|
||||||
|
\begin{titlepage}
|
||||||
|
|
||||||
|
\centering
|
||||||
|
{\huge\bfseries \par}
|
||||||
|
\vspace{2cm}
|
||||||
|
{\LARGE\itshape Simon Kluettermann\par}
|
||||||
|
\vspace{1.5cm}
|
||||||
|
{\scshape\Large Master Thesis in Physics\par}
|
||||||
|
\vspace{0.2cm}
|
||||||
|
{\Large submitted to the \par}
|
||||||
|
\vspace{0.2cm}
|
||||||
|
{\scshape\Large Faculty of Mathematics Computer Science and Natural Sciences \par}
|
||||||
|
\vspace{0.2cm}
|
||||||
|
{\Large \par}
|
||||||
|
\vspace{0.2cm}
|
||||||
|
{\scshape\Large RWTH Aachen University}
|
||||||
|
\vspace{1cm}
|
||||||
|
|
||||||
|
\vfill
|
||||||
|
{\scshape\Large Department of Physics\par}
|
||||||
|
\vspace{0.2cm}
|
||||||
|
{\scshape\Large Insitute for theoretical Particle Physics and Cosmology\par}
|
||||||
|
\vspace{0.2cm}
|
||||||
|
{ \Large\par}
|
||||||
|
\vspace{0.2cm}
|
||||||
|
{\Large First Referee: Prof. Dr. Michael Kraemer \par}
|
||||||
|
{\Large Second Referee: Prof. Dr. Felix Kahlhoefer}
|
||||||
|
|
||||||
|
\vfill
|
||||||
|
|
||||||
|
% Bottom of the page
|
||||||
|
{\large November 2020 \par}
|
||||||
|
\end{titlepage}
|
||||||
|
\pagenumbering{roman}
|
||||||
|
\thispagestyle{empty}
|
||||||
|
\null
|
||||||
|
\newpage
|
||||||
|
\setcounter{page}{1}
|
||||||
|
\pagenumbering{arabic}
|
||||||
|
\end{frame}
|
||||||
|
|
||||||
|
|
||||||
|
%from file ../case3/data/001reminder.txt
|
||||||
|
\begin{frame}[label=reminder]
|
||||||
|
\frametitle{reminder}
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item No Case Study next week
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item neither Tuesday (29.11) nor Thursday (01.12)
|
||||||
|
|
||||||
|
\item if you need help: just write me an email!
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\item In two weeks: Case Study switched
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Q+A Tuesday (6.12, 14:00) online only
|
||||||
|
|
||||||
|
\item Case Study Meeting Thursday (08.12, 14:00-16:00), in OH12 Room 3.032
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\end{frame}
|
||||||
|
|
||||||
|
|
||||||
|
%from file ../case3/data/002Big Picture.txt
|
||||||
|
\begin{frame}[label=Big Picture]
|
||||||
|
\frametitle{Big Picture}
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Goal for this case study: Have better hyperparameters than pyod!
|
||||||
|
|
||||||
|
\item So each of you: Gets assigned two algorithms!
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item One fairly simple before
|
||||||
|
|
||||||
|
\item One more complicated one today
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\item Try to find the best possible hyperparameters for your algorithms
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Try to be clever (for example: PCA: $n_{components}<n_{features}$. Maybe $\frac{n_{components}}{n_{features}}$ constant?
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\item Afterwards
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Write down your findings into a simple function (given data, what are my best hyperparameters)
|
||||||
|
|
||||||
|
\item Write down your finding into a report (together, double collumn. 6 Pages per student, plus comparison of algorithms to each other)
|
||||||
|
|
||||||
|
\item One final presentation together in front of my colleagues. About 10min per student.
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\end{frame}
|
||||||
|
|
||||||
|
|
||||||
|
%from file ../case3/data/003Evaluating your hyperparameter.txt
|
||||||
|
\begin{frame}[label=Evaluating your hyperparameter]
|
||||||
|
\frametitle{Evaluating your hyperparameter}
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item My suggestion: Compare to normal parameters.
|
||||||
|
|
||||||
|
\item This means you get two lists of AUC scores
|
||||||
|
|
||||||
|
\item Your params: [0.80,0.75,0.73,....,0.95]
|
||||||
|
|
||||||
|
\item Pyod params: [0.82,0.71,0.48,....,0.95]
|
||||||
|
|
||||||
|
\item look at two values
|
||||||
|
|
||||||
|
\item $\sum_i your_i-pyod_i$
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Total improvment. If positive, then your parameters help;)
|
||||||
|
|
||||||
|
\item But hard to see if this is significant
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\item Fraction of $your_i>pyod_i$
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Quantised, so does not care about improving your parameters further
|
||||||
|
|
||||||
|
\item But easy to see if this is significant
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item 0.5$\Rightarrow$Probably just random
|
||||||
|
|
||||||
|
\item 0.9$\Rightarrow$Probably quite significant
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\end{frame}
|
||||||
|
|
||||||
|
|
||||||
|
%from file ../case3/data/004How to continue.txt
|
||||||
|
\begin{frame}[label=How to continue]
|
||||||
|
\frametitle{How to continue}
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item See how far you can improve this?
|
||||||
|
|
||||||
|
\item Treat this as a supervised optimisation problem: Given this dataset, find the best hyperparameters
|
||||||
|
|
||||||
|
\item Might be useful to look at more input parameters
|
||||||
|
|
||||||
|
\item Might help to formulate your parameters differently
|
||||||
|
|
||||||
|
\item But be aware of \textbf{overfitting}!
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\end{frame}
|
||||||
|
|
||||||
|
|
||||||
|
%from file ../case3/data/005Intro to Deep Learning.txt
|
||||||
|
\begin{frame}[label=Intro to Deep Learning]
|
||||||
|
\frametitle{Intro to Deep Learning}
|
||||||
|
\begin{figure}[H]
|
||||||
|
\centering
|
||||||
|
\includegraphics[width=0.9\textwidth]{..//prep/05Intro_to_Deep_Learning/adsasda.png}
|
||||||
|
\label{fig:prep05Intro_to_Deep_Learningadsasdapng}
|
||||||
|
\end{figure}
|
||||||
|
|
||||||
|
|
||||||
|
\end{frame}
|
||||||
|
|
||||||
|
|
||||||
|
%from file ../case3/data/006Intro to Deep Learning.txt
|
||||||
|
\begin{frame}[label=Intro to Deep Learning]
|
||||||
|
\frametitle{Intro to Deep Learning}
|
||||||
|
\begin{figure}[H]
|
||||||
|
\centering
|
||||||
|
\includegraphics[width=0.9\textwidth]{..//prep/06Intro_to_Deep_Learning/adsasd.png}
|
||||||
|
\label{fig:prep06Intro_to_Deep_Learningadsasdpng}
|
||||||
|
\end{figure}
|
||||||
|
|
||||||
|
|
||||||
|
\end{frame}
|
||||||
|
|
||||||
|
|
||||||
|
%from file ../case3/data/007Intro to Deep Learning.txt
|
||||||
|
\begin{frame}[label=Intro to Deep Learning]
|
||||||
|
\frametitle{Intro to Deep Learning}
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item The idea is always the same:
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Define complicated model to learn (often millions of parameters)
|
||||||
|
|
||||||
|
\item Define loss function that this model should minimize (example: $\sum_i (y_i-f(x_i))^2$)
|
||||||
|
|
||||||
|
\item Find parameters that minimize the loss ($\Rightarrow$Backpropagation)
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\item Usually Neural Networks:
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item $f(x)=f_n(x)=activation(A_n\cdot f_{n-1}(x)+b_n)$
|
||||||
|
|
||||||
|
\item $f_0(x)=x$
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\item Powerful, as you can show that when there are 3 Layers+ (and infinitely sized matrices), you can approximate any function
|
||||||
|
|
||||||
|
\item $\Rightarrow$So a model becomes a loss function
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\end{frame}
|
||||||
|
|
||||||
|
|
||||||
|
%from file ../case3/data/008Autoencoder.txt
|
||||||
|
\begin{frame}[label=Autoencoder]
|
||||||
|
\frametitle{Autoencoder}
|
||||||
|
\begin{figure}[H]
|
||||||
|
\centering
|
||||||
|
\includegraphics[width=0.9\textwidth]{..//prep/08Autoencoder/ae.png}
|
||||||
|
\label{fig:prep08Autoencoderaepng}
|
||||||
|
\end{figure}
|
||||||
|
|
||||||
|
|
||||||
|
\end{frame}
|
||||||
|
|
||||||
|
|
||||||
|
%from file ../case3/data/009Autoencoder.txt
|
||||||
|
\begin{frame}[label=Autoencoder]
|
||||||
|
\frametitle{Autoencoder}
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Lets look at some of its Hyperparameters
|
||||||
|
|
||||||
|
\item Autoencoder Specific
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Compression factor (Latent space size)
|
||||||
|
|
||||||
|
\item Loss function (mse?)
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\item Neural Network architecture
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Number of layers
|
||||||
|
|
||||||
|
\item Number of neurons in each layer (Shape of the matrices $A_n$)
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\item Optimisation parameters
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Learning Rate
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Controls how fast the parameters are found
|
||||||
|
|
||||||
|
\item To high value makes the training unstable
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\item Batch size
|
||||||
|
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item Controls how many samples are averaged together.
|
||||||
|
|
||||||
|
\item Lower values make the training more stable, but also the result less optimal
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\end{frame}
|
||||||
|
|
||||||
|
|
||||||
|
%from file ../case3/data/010For next time.txt
|
||||||
|
\begin{frame}[label=For next time]
|
||||||
|
\frametitle{For next time}
|
||||||
|
\begin{itemize}
|
||||||
|
|
||||||
|
\item (if you have not finished finding good parameters for your old algorithm, continue searching for them)
|
||||||
|
|
||||||
|
\item Take a look at your new algorithm
|
||||||
|
|
||||||
|
\item Run it once on cardio, take a look at which parameters you have
|
||||||
|
|
||||||
|
\item Prepare a similar presentation to last time (include your cardio result)
|
||||||
|
|
||||||
|
|
||||||
|
\end{itemize}
|
||||||
|
\end{frame}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
\end{document}
|
|
@ -0,0 +1 @@
|
||||||
|
<titlepage>
|
|
@ -0,0 +1,11 @@
|
||||||
|
No Case Study next week
|
||||||
|
<l2st>
|
||||||
|
neither Tuesday (29.11) nor Thursday (01.12)
|
||||||
|
if you need help: just write me an email!
|
||||||
|
</l2st>
|
||||||
|
In two weeks: Case Study switched
|
||||||
|
<l2st>
|
||||||
|
Q+A Tuesday (6.12, 14:00) online only
|
||||||
|
Case Study Meeting Thursday (08.12, 14:00-16:00), in OH12 Room 3.032
|
||||||
|
</l2st>
|
||||||
|
|
|
@ -0,0 +1,16 @@
|
||||||
|
Goal for this case study: Have better hyperparameters than pyod!
|
||||||
|
So each of you: Gets assigned two algorithms!
|
||||||
|
<l2st>
|
||||||
|
One fairly simple before
|
||||||
|
One more complicated one today
|
||||||
|
</l2st>
|
||||||
|
Try to find the best possible hyperparameters for your algorithms
|
||||||
|
<l2st>
|
||||||
|
Try to be clever (for example: PCA: $n_{components}<n_{features}$. Maybe $\frac{n_{components}}{n_{features}}$ constant?
|
||||||
|
</l2st>
|
||||||
|
Afterwards
|
||||||
|
<l2st>
|
||||||
|
Write down your findings into a simple function (given data, what are my best hyperparameters)
|
||||||
|
Write down your finding into a report (together, double collumn. 6 Pages per student, plus comparison of algorithms to each other)
|
||||||
|
One final presentation together in front of my colleagues. About 10min per student.
|
||||||
|
</l2st>
|
|
@ -0,0 +1,21 @@
|
||||||
|
My suggestion: Compare to normal parameters.
|
||||||
|
This means you get two lists of AUC scores
|
||||||
|
Your params: [0.80,0.75,0.73,....,0.95]
|
||||||
|
Pyod params: [0.82,0.71,0.48,....,0.95]
|
||||||
|
look at two values
|
||||||
|
$\sum_i your_i-pyod_i$
|
||||||
|
<l2st>
|
||||||
|
Total improvment. If positive, then your parameters help;)
|
||||||
|
But hard to see if this is significant
|
||||||
|
</l2st>
|
||||||
|
Fraction of $your_i>pyod_i$
|
||||||
|
<l2st>
|
||||||
|
Quantised, so does not care about improving your parameters further
|
||||||
|
But easy to see if this is significant
|
||||||
|
<l3st>
|
||||||
|
0.5->Probably just random
|
||||||
|
0.9->Probably quite significant
|
||||||
|
</l3st>
|
||||||
|
</l2st>
|
||||||
|
|
||||||
|
|
|
@ -0,0 +1,5 @@
|
||||||
|
See how far you can improve this?
|
||||||
|
Treat this as a supervised optimisation problem: Given this dataset, find the best hyperparameters
|
||||||
|
Might be useful to look at more input parameters
|
||||||
|
Might help to formulate your parameters differently
|
||||||
|
But be aware of \textbf{overfitting}!
|
After Width: | Height: | Size: 264 KiB |
After Width: | Height: | Size: 182 KiB |
|
@ -0,0 +1,16 @@
|
||||||
|
The idea is always the same:
|
||||||
|
<l2st>
|
||||||
|
Define complicated model to learn (often millions of parameters)
|
||||||
|
Define loss function that this model should minimize (example: $\sum_i (y_i-f(x_i))^2$)
|
||||||
|
Find parameters that minimize the loss (->Backpropagation)
|
||||||
|
</l2st>
|
||||||
|
Usually Neural Networks:
|
||||||
|
<l2st>
|
||||||
|
$f(x)=f_n(x)=activation(A_n\cdot f_{n-1}(x)+b_n)$
|
||||||
|
$f_0(x)=x$
|
||||||
|
</l2st>
|
||||||
|
Powerful, as you can show that when there are 3 Layers+ (and infinitely sized matrices), you can approximate any function
|
||||||
|
->So a model becomes a loss function
|
||||||
|
|
||||||
|
|
||||||
|
|
After Width: | Height: | Size: 260 KiB |
|
@ -0,0 +1,24 @@
|
||||||
|
Lets look at some of its Hyperparameters
|
||||||
|
Autoencoder Specific
|
||||||
|
<l2st>
|
||||||
|
Compression factor (Latent space size)
|
||||||
|
Loss function (mse?)
|
||||||
|
</l2st>
|
||||||
|
Neural Network architecture
|
||||||
|
<l2st>
|
||||||
|
Number of layers
|
||||||
|
Number of neurons in each layer (Shape of the matrices $A_n$)
|
||||||
|
</l2st>
|
||||||
|
Optimisation parameters
|
||||||
|
<l2st>
|
||||||
|
Learning Rate
|
||||||
|
<l3st>
|
||||||
|
Controls how fast the parameters are found
|
||||||
|
To high value makes the training unstable
|
||||||
|
</l3st>
|
||||||
|
Batch size
|
||||||
|
<l3st>
|
||||||
|
Controls how many samples are averaged together.
|
||||||
|
Lower values make the training more stable, but also the result less optimal
|
||||||
|
</l3st>
|
||||||
|
</l2st>
|
|
@ -0,0 +1,4 @@
|
||||||
|
(if you have not finished finding good parameters for your old algorithm, continue searching for them)
|
||||||
|
Take a look at your new algorithm
|
||||||
|
Run it once on cardio, take a look at which parameters you have
|
||||||
|
Prepare a similar presentation to last time (include your cardio result)
|
After Width: | Height: | Size: 20 KiB |
After Width: | Height: | Size: 989 KiB |
After Width: | Height: | Size: 265 KiB |