-
Notifications
You must be signed in to change notification settings - Fork 0
/
biosciences-math-fundamentals.html
1587 lines (1587 loc) · 145 KB
/
biosciences-math-fundamentals.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
<head>
<meta charset="utf-8" />
<meta name="generator" content="pandoc" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
<meta name="author" content="Samuel Bernard" />
<meta name="author" content="Laurent Pujo-Menjouet" />
<title>Elements of Maths for Biology</title>
<style>
html {
line-height: 1.5;
font-family: Georgia, serif;
font-size: 20px;
color: #1a1a1a;
background-color: #fdfdfd;
}
body {
margin: 0 auto;
max-width: 36em;
padding-left: 50px;
padding-right: 50px;
padding-top: 50px;
padding-bottom: 50px;
hyphens: auto;
overflow-wrap: break-word;
text-rendering: optimizeLegibility;
font-kerning: normal;
}
@media (max-width: 600px) {
body {
font-size: 0.9em;
padding: 1em;
}
}
@media print {
body {
background-color: transparent;
color: black;
font-size: 12pt;
}
p, h2, h3 {
orphans: 3;
widows: 3;
}
h2, h3, h4 {
page-break-after: avoid;
}
}
p {
margin: 1em 0;
}
a {
color: #1a1a1a;
}
a:visited {
color: #1a1a1a;
}
img {
max-width: 100%;
}
h1, h2, h3, h4, h5, h6 {
margin-top: 1.4em;
}
h5, h6 {
font-size: 1em;
font-style: italic;
}
h6 {
font-weight: normal;
}
ol, ul {
padding-left: 1.7em;
margin-top: 1em;
}
li > ol, li > ul {
margin-top: 0;
}
blockquote {
margin: 1em 0 1em 1.7em;
padding-left: 1em;
border-left: 2px solid #e6e6e6;
color: #606060;
}
code {
font-family: Menlo, Monaco, 'Lucida Console', Consolas, monospace;
font-size: 85%;
margin: 0;
}
pre {
margin: 1em 0;
overflow: auto;
}
pre code {
padding: 0;
overflow: visible;
overflow-wrap: normal;
}
.sourceCode {
background-color: transparent;
overflow: visible;
}
hr {
background-color: #1a1a1a;
border: none;
height: 1px;
margin: 1em 0;
}
table {
margin: 1em 0;
border-collapse: collapse;
width: 100%;
overflow-x: auto;
display: block;
font-variant-numeric: lining-nums tabular-nums;
}
table caption {
margin-bottom: 0.75em;
}
tbody {
margin-top: 0.5em;
border-top: 1px solid #1a1a1a;
border-bottom: 1px solid #1a1a1a;
}
th {
border-top: 1px solid #1a1a1a;
padding: 0.25em 0.5em 0.25em 0.5em;
}
td {
padding: 0.125em 0.5em 0.25em 0.5em;
}
header {
margin-bottom: 4em;
text-align: center;
}
#TOC li {
list-style: none;
}
#TOC a:not(:hover) {
text-decoration: none;
}
code{white-space: pre-wrap;}
span.smallcaps{font-variant: small-caps;}
span.underline{text-decoration: underline;}
div.column{display: inline-block; vertical-align: top; width: 50%;}
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
ul.task-list{list-style: none;}
</style>
<script src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-chtml-full.js" type="text/javascript"></script>
<!--[if lt IE 9]>
<script src="//cdnjs.cloudflare.com/ajax/libs/html5shiv/3.7.3/html5shiv-printshiv.min.js"></script>
<![endif]-->
</head>
<body>
<header id="title-block-header">
<h1 class="title">Elements of Maths for Biology</h1>
<p class="author">Samuel Bernard<a href="#fn1" class="footnote-ref" id="fnref1" role="doc-noteref"><sup>1</sup></a></p>
<p class="author">Laurent Pujo-Menjouet<a href="#fn2" class="footnote-ref" id="fnref2" role="doc-noteref"><sup>2</sup></a></p>
<p class="date">V2022.05.24</p>
</header>
<nav id="TOC" role="doc-toc">
<ul>
<li><a href="#readme">README</a></li>
<li><a href="#fonctions-maps">Fonctions, maps</a>
<ul>
<li><a href="#some-usual-maps">Some usual maps</a></li>
<li><a href="#exercises-on-functions">Exercises on functions</a></li>
</ul></li>
<li><a href="#derivatives">Derivatives</a>
<ul>
<li><a href="#list-of-common-derivatives">List of common derivatives</a></li>
<li><a href="#exercises-on-derivatives">Exercises on derivatives</a></li>
</ul></li>
<li><a href="#taylor-series-and-truncated-expansions">Taylor series and truncated expansions</a>
<ul>
<li><a href="#expansion-of-a-function-of-two-variables">Expansion of a function of two variables</a></li>
<li><a href="#expansion-of-a-function-from-mathbbr2-to-mathbbr2">Expansion of a function from <span class="math inline">\(\mathbb{R}^2 \to \mathbb{R}^2\)</span></a></li>
</ul></li>
<li><a href="#integrals-and-primitives">Integrals and primitives</a>
<ul>
<li><a href="#primitives">Primitives</a></li>
<li><a href="#integrals">Integrals</a></li>
</ul></li>
<li><a href="#differential-equations-in-1d">Differential equations in 1D</a>
<ul>
<li><a href="#finding-solutions-of-differential-equations">Finding solutions of differential equations</a></li>
</ul></li>
<li><a href="#complex-numbers">Complex numbers</a>
<ul>
<li><a href="#roots-of-a-complex-number">Roots of a complex number</a></li>
<li><a href="#exercises-on-complex-numbers">Exercises on complex numbers</a></li>
</ul></li>
<li><a href="#matrices-in-dimension-2">Matrices in dimension 2</a>
<ul>
<li><a href="#eigenvalues-of-a-2-times-2-matrix">Eigenvalues of a <span class="math inline">\(2 \times 2\)</span> matrix</a>
<ul>
<li><a href="#exercises-on-eigenvalues">Exercises on eigenvalues</a></li>
</ul></li>
<li><a href="#matrix-vector-operations">Matrix-vector operations </a>
<ul>
<li><a href="#exercises-on-matrix-vector-and-matrix-matrix-operations">Exercises on Matrix-vector and matrix-matrix operations</a></li>
</ul></li>
</ul></li>
<li><a href="#eigenvalue-decomposition">Eigenvalue decomposition</a>
<ul>
<li><a href="#eigenvectors">Eigenvectors</a></li>
<li><a href="#exercises-on-eigenvalues-decomposition">Exercises on eigenvalues decomposition</a></li>
</ul></li>
<li><a href="#linearisation-of-functions-mathbbr2-to-mathbbr2">Linearisation of functions <span class="math inline">\(\mathbb{R}^2 \to \mathbb{R}^2\)</span></a>
<ul>
<li><a href="#exercises-on-linearisation">Exercises on linearisation </a></li>
</ul></li>
<li><a href="#solution-of-systems-of-linear-differential-equations-in-dimension-2">Solution of systems of linear differential equations in dimension 2</a></li>
<li><a href="#glossary">Glossary</a></li>
</ul>
</nav>
<h1 id="readme">README</h1>
<p>This document is intended for training and future reference. As a reference document, you may find it useful for Biomaths 1 (3BS, Fall semester), for linear algebra (3BIM Biomaths 2, Winter Semester) and for Advanced ODEs (3BIM Biomaths 3, Winter Semester).</p>
<p>When important concepts are encoutered for the first time, they are highlighted in <strong>bold</strong> next to their definition. We tried to provides examples that are as complete as possible. This means that they are long, you could probably solve them faster. Exercises are important, they can introduce theory or techniques that will be prove useful. Solutions to the exercises can be found at the end of the documents. Examples and exercises (and their solutions!) will be added regularly, so feel free download the most recent version (<a href="https://moodle.insa-lyon.fr/pluginfile.php/306298/mod_resource/content/3/insa-biosciences-remise-a-niveau-math.pdf">here</a>). This is version V2022.05.24.</p>
<h1 id="fonctions-maps">Fonctions, maps</h1>
<p>A <strong>function</strong> is a relation, often denoted <span class="math inline">\(f\)</span>, that associates an element <span class="math inline">\(x\)</span> of a <strong>domain</strong> <span class="math inline">\(I\)</span>, and at most one element <span class="math inline">\(y\)</span> of the <strong>image</strong> <span class="math inline">\(J\)</span>. The domain <span class="math inline">\(I\)</span> and image <span class="math inline">\(J\)</span> are sets. Usually <span class="math inline">\(I, J \in \mathbb{R}\)</span>.</p>
<figure>
<img src="htmltmp/tikzblocks_0.pdf.png" id="f_functions" alt="Functions. (A) Function f. (B) Not a function. " /><figcaption aria-hidden="true">Functions. (A) Function <span class="math inline">\(f\)</span>. (B) Not a function. </figcaption>
</figure>
<p>A <strong>map</strong> is a relation that associate <em>each</em> element of its domain to exactly one element of its image. Maps and functions are related but slightly different concepts. A function <span class="math inline">\(f\)</span> is a map if it is defined for all elements of its domain <span class="math inline">\(I\)</span>. A map is always a function. The term <em>map</em> can also be used when the domain or the image are not numbers (Figure <a href="#f_functions" data-reference-type="ref" data-reference="f_functions">1</a>).</p>
<p>The <strong>graph</strong> of a function <span class="math inline">\(f\)</span>, denoted <span class="math inline">\(\mathcal{G}(f)\)</span> is the set of all pairs <span class="math inline">\((x, f(x))\)</span> in the <span class="math inline">\(I \times J\)</span> plane. For real-valued functions, the graph is represented as a curve in the Cartesian plane.</p>
<p>Functions are not numbers. Do not confuse</p>
<ul>
<li><p><span class="math inline">\(f\)</span> the function</p></li>
<li><p><span class="math inline">\(f(x)\)</span> the evaluation of <span class="math inline">\(f\)</span> at element <span class="math inline">\(x\)</span>; <span class="math inline">\(f(x)\)</span> is an element of the image (usually a number)</p></li>
<li><p><span class="math inline">\(\mathcal{G}(f)\)</span> the graph of <span class="math inline">\(f\)</span></p></li>
</ul>
<p>Consequently, do not write</p>
<ul>
<li><p><span class="math inline">\(f(x)\)</span> is increasing... Instead write <span class="math inline">\(f\)</span> is increasing...</p></li>
<li><p><span class="math inline">\(f(x)\)</span> is decreasing... Instead write <span class="math inline">\(f\)</span> is decreasing...</p></li>
<li><p><span class="math inline">\(f(x)\)</span> is continuous... Instead write <span class="math inline">\(f\)</span> continuous...</p></li>
</ul>
<h2 id="some-usual-maps">Some usual maps</h2>
<ul>
<li><p><span class="math inline">\(f:\mathbb{R} \to \mathbb{R}\)</span>, with <span class="math inline">\(x \to k\)</span>, <span class="math inline">\(k \in \mathbb{R}\)</span> constant; <span class="math inline">\(x \to x\)</span>, <strong>identity map</strong>.</p>
<div class="center">
<p><img src="htmltmp/tikzblocks_1.pdf.png" alt="image" /> <img src="htmltmp/tikzblocks_2.pdf.png" alt="image" /></p>
</div></li>
<li><p><span class="math inline">\(f:\mathbb{R}\backslash\{0\} \to \mathbb{R}\)</span>, with <span class="math inline">\(x \to \frac 1x\)</span>, <strong>inverse</strong>.</p>
<div class="center">
<p><img src="htmltmp/tikzblocks_3.pdf.png" alt="image" /></p>
</div></li>
<li><p><span class="math inline">\(f:\mathbb{R} \to \mathbb{R}\)</span>, with <span class="math inline">\(x \to x^2\)</span>, parabola; <span class="math inline">\(x \to x^3\)</span>, <strong>cubic map</strong>.</p>
<div class="center">
<p><img src="htmltmp/tikzblocks_4.pdf.png" alt="image" /> <img src="htmltmp/tikzblocks_5.pdf.png" alt="image" /></p>
</div></li>
<li><p><span class="math inline">\(f:\mathbb{R}^+ \to \mathbb{R}\)</span>, with <span class="math inline">\(x \to \sqrt{x} = x^{\frac 12}\)</span>, <strong>square root</strong>; more generally with <span class="math inline">\(x \to x^{\frac pq} = {}^q\sqrt{x^p}\)</span>, <strong>fractional power</strong>.</p>
<div class="center">
<p><img src="htmltmp/tikzblocks_6.pdf.png" alt="image" /> <img src="htmltmp/tikzblocks_7.pdf.png" alt="image" /></p>
</div></li>
<li><p><span class="math inline">\(f:\mathbb{R}\backslash\{-d/c\} \to \mathbb{R}\)</span>, with <span class="math inline">\(x \to \frac{ax + b}{cx + d}\)</span>.</p>
<div class="center">
<p><img src="htmltmp/tikzblocks_8.pdf.png" alt="image" /></p>
</div></li>
<li><p><span class="math inline">\(f:\mathbb{R} \to \mathbb{R}\)</span>, with <span class="math inline">\(x \to \exp(x)\)</span>, <strong>exponential</strong>.</p>
<div class="center">
<p><img src="htmltmp/tikzblocks_9.pdf.png" alt="image" /></p>
</div></li>
<li><p><span class="math inline">\(f:\mathbb{R}^+\backslash\{0\} \to \mathbb{R}\)</span>, with <span class="math inline">\(x \to \ln(x)\)</span>, <strong>natural logarithm</strong>.</p>
<div class="center">
<p><img src="htmltmp/tikzblocks_10.pdf.png" alt="image" /></p>
</div>
<p>On logarithms: For <span class="math inline">\(a, b >0\)</span>, <span class="math inline">\(n\)</span> positive integer, <span class="math inline">\(\ln(ab) = \ln(a) + \ln(b)\)</span>, <span class="math inline">\(\ln(a^n) = n \ln(a)\)</span>, <span class="math inline">\(\ln(a/b) = \ln(a) - \ln(b)\)</span>.</p></li>
<li><p><span class="math inline">\(f:\mathbb{R} \to \mathbb{R}\)</span>, with <span class="math inline">\(x \to \cos(x)\)</span>, <strong>cosine</strong>; <span class="math inline">\(x \to \sin(x)\)</span>, <strong>sine</strong> <span class="math inline">\(x \to \tan(x)\)</span>, <strong>tangent</strong>.</p>
<div class="center">
<p><img src="htmltmp/tikzblocks_11.pdf.png" alt="image" /> <img src="htmltmp/tikzblocks_12.pdf.png" alt="image" /> <img src="htmltmp/tikzblocks_13.pdf.png" alt="image" /></p>
</div>
<p>A bit more on trigonometric functions. The diagram below showis the relationship between the sine, cosine and tangent, of an angle <span class="math inline">\(\theta \in [0,2\pi]\)</span>.</p>
<div class="center">
<p><img src="htmltmp/tikzblocks_14.pdf.png" alt="image" /></p>
</div></li>
<li><p><span class="math inline">\(f:\mathbb{R} \to \mathbb{R}\)</span>, with <span class="math inline">\(x \to \cosh(x) = \frac 12 \bigl( e^{x} + e^{-x} \bigr)\)</span>, <strong>hyperbolic cosine</strong>; <span class="math inline">\(x \to \sinh(x) = \frac 12 \bigl( e^{x} - e^{-x} \bigr)\)</span>, <strong>hyperbolic sine</strong>; <span class="math inline">\(x \to \tanh(x) = \frac{\sinh(x)}{\cosh(x)}\)</span>, <strong>hyperbolic tangent</strong>.</p>
<div class="center">
<p><img src="htmltmp/tikzblocks_15.pdf.png" alt="image" /> <img src="htmltmp/tikzblocks_16.pdf.png" alt="image" /> <img src="htmltmp/tikzblocks_17.pdf.png" alt="image" /></p>
</div></li>
</ul>
<h2 id="exercises-on-functions">Exercises on functions</h2>
<p>More to come...</p>
<h1 id="derivatives">Derivatives</h1>
<p>We call the <strong>derivative</strong> of the function <span class="math inline">\(f:I \to J\)</span> (<span class="math inline">\(I,J \subset \mathbb{R}),\)</span>, at point <span class="math inline">\(a \in I\)</span> the limit, if it exists, <span class="math display">\[\lim_{x \to a} \frac{f(x) - f(a)}{x - a}.\]</span> The derivative is denoted <span class="math inline">\(f'(a)\)</span>. An alternative representation of the limit is obtained by setting <span class="math inline">\(h = x - a\)</span>, <span class="math display">\[f'(a) = \lim_{h \to 0} \frac{f(a + h) - f(a)}{h}.\]</span> If the derivative exists for all elements <span class="math inline">\(a \in I\)</span>, we say that <strong>differentiable</strong> on <span class="math inline">\(I\)</span>.</p>
<ul>
<li><p>If <span class="math inline">\(f\)</span> is differentiable on <span class="math inline">\(I\)</span>, and <span class="math inline">\(f'(x) > 0\)</span>, then <span class="math inline">\(f\)</span> is strictly increasing on <span class="math inline">\(I\)</span>.</p></li>
<li><p>If <span class="math inline">\(f\)</span> is differentiable on <span class="math inline">\(I\)</span>, and <span class="math inline">\(f'(x) < 0\)</span>, then <span class="math inline">\(f\)</span> is strictly decreasing on <span class="math inline">\(I\)</span>.</p></li>
</ul>
<p>However, if <span class="math inline">\(f\)</span> is strictly increasing, it does not mean that <span class="math inline">\(f'(x) > 0\)</span>. For example the function <span class="math inline">\(f\)</span> with <span class="math inline">\(f(x) = x^3\)</span> is strictly increasing on <span class="math inline">\(\mathbb{R}\)</span>, but <span class="math inline">\(f'(0) = 0\)</span>. Where the derivative exists, we can define the derivative function <span class="math inline">\(f':I \to \mathbb{R}\)</span> of <span class="math inline">\(f\)</span>.</p>
<p>The <strong>second derivative</strong> of a function <span class="math inline">\(f\)</span>, denoted <span class="math inline">\(f''\)</span> is the derivative of <span class="math inline">\(f'\)</span>, where defined. If <span class="math inline">\(f''(x)\)</span> exists and <span class="math inline">\(f''(x) > 0\)</span> for all <span class="math inline">\(x \in I\)</span>, we say that <span class="math inline">\(f\)</span> is <strong>convex</strong> (U-shaped). If <span class="math inline">\(f'(x) = 0\)</span> and <span class="math inline">\(f''(x) > 0\)</span>, the point <span class="math inline">\(x\)</span> is a <strong>minimum</strong>. If <span class="math inline">\(f'(x)\)</span> and <span class="math inline">\(f''(x) < 0\)</span>, the point <span class="math inline">\(x\)</span> is a <strong>maxmimum</strong>. Maxima and minima are <strong>extrema</strong>. If <span class="math inline">\(f''(0) = 0\)</span>, the point <span class="math inline">\(x\)</span> is an <strong>inflection point</strong> (Figure <a href="#f_extrema" data-reference-type="ref" data-reference="f_extrema">2</a>).</p>
<figure>
<img src="htmltmp/tikzblocks_18.pdf.png" id="f_extrema" alt="Extrema, inflection points of the polynomial f(x) = (x+0.8)(x-0.5)(x-0.8). " /><figcaption aria-hidden="true">Extrema, inflection points of the polynomial <span class="math inline">\(f(x) = (x+0.8)(x-0.5)(x-0.8).\)</span> </figcaption>
</figure>
<h2 id="list-of-common-derivatives">List of common derivatives</h2>
<p>The derivative has linear properties. If <span class="math inline">\(f\)</span> and <span class="math inline">\(g\)</span> are differentiable on <span class="math inline">\(I\)</span>, and <span class="math inline">\(a \in \mathbb{R}\)</span>,</p>
<ul>
<li><p><span class="math inline">\((f + g)' = f' + g'\)</span>.</p></li>
<li><p><span class="math inline">\((af)' = a (f')\)</span>.</p></li>
<li><p><span class="math inline">\((af + g)' = a (f') + g'\)</span>.</p></li>
</ul>
<p>Let <span class="math inline">\(g:I \to J\)</span> and <span class="math inline">\(f:J \to K\)</span> be two functions. The <strong>composition</strong> of <span class="math inline">\(f\)</span> and <span class="math inline">\(g\)</span>, denoted <span class="math inline">\(f \circ g\)</span>, is the function <span class="math inline">\(x \to f(g(x))\)</span>, i.e. <span class="math inline">\(f \circ g (x) = f(g(x))\)</span>. If <span class="math inline">\(f\)</span> and <span class="math inline">\(g\)</span> are differentiable, the composition <span class="math inline">\(f \circ g\)</span> is also differentiable, and its derivative follows the <strong>rule of composed functions</strong>:</p>
<p><span class="math display">\[\bigl( f \circ g \bigr)'(x) = f'(g(x))g'(x).\]</span></p>
<p><strong>Example </strong> Let <span class="math inline">\(f:x \to x^2\)</span> and <span class="math inline">\(g:x \to 3x + 1\)</span> be two differentiable functions, with <span class="math inline">\(f'(x) = 2x\)</span> and <span class="math inline">\(g'(x) = 3\)</span>. The derivative of the composed function <span class="math inline">\(f \circ g\)</span> at <span class="math inline">\(x\)</span> is <span class="math display">\[f'(g(x))g'(x) = f'(3x+1)g'(x) = 2(3x+1) \cdot 3 = 6(3x+1) = 18x + 6.\]</span> The derivative could have been obtained by first computing the composed function <span class="math inline">\(f(g(x)) = (3x+1)^2 =
9 x^2 + 6x + 1\)</span>, and then taking teh derivative.</p>
<p><strong>Example </strong> Compute the derivative of <span class="math inline">\(f: x \to \sin(1/x)\)</span>. The function <span class="math inline">\(f\)</span> is composed of a sine and an inverse function. To compute the derivative, we decomposed the function <span class="math inline">\(f\)</span> as <span class="math inline">\(f(x) = g(h(x))\)</span> with <span class="math inline">\(g(x) = \sin(x)\)</span> and <span class="math inline">\(h(x) = 1/x\)</span>. The derivatives are <span class="math inline">\(g'(x) = \cos(x)\)</span> and <span class="math inline">\(h'(x) = -1/x^2\)</span>. Finally the derivative of <span class="math inline">\(f\)</span> is</p>
<p><span class="math display">\[f'(x) = g'(h(x))h'(x) = \cos(1/x) \Bigl( \frac{-1}{x^2} \Bigr) = - \frac{\cos(1/x)}{x^2}.\]</span></p>
<p><strong>Example </strong> A function <span class="math inline">\(f:I \to I\)</span> is bijective (<strong>invertible</strong>) on <span class="math inline">\(I\)</span> if there exists a function, denoted <span class="math inline">\(f^{-1}\)</span> and called <strong>inverse</strong> of <span class="math inline">\(f\)</span>, such that the compositions are equal <span class="math inline">\(f \circ f^{-1} = f^{-1} \circ f\)</span>, and are equal to the identity map. That is, <span class="math inline">\(f^{-1} \circ f (x) = f \circ f^{-1}(x) = x\)</span> for all <span class="math inline">\(x \in I\)</span>. If <span class="math inline">\(f\)</span> is differentiable and invertible, what is the derivative of <span class="math inline">\(f^{-1}\)</span>?</p>
<p>We apply the derivative to <span class="math inline">\(f(f^{-1})\)</span>. Given that <span class="math inline">\(f(f^{-1}(x)) = x\)</span> by definition, we have <span class="math inline">\(\bigl( f(f^{-1}) \bigr)' = 1,\)</span> and <span class="math display">\[\begin{aligned}
\bigl( f(f^{-1}) \bigr)'(x) & = f'(f^{-1}(x))(f^{-1})'(x), \\
& = 1, \\
(f^{-1})'(x) & = \frac{1}{f'(f^{-1}(x))}.\end{aligned}\]</span> Take for instance <span class="math inline">\(f(x) = x^2\)</span> on <span class="math inline">\(x \in (0,1]\)</span>. The inverse of <span class="math inline">\(f\)</span> is <span class="math inline">\(f^{-1}(x) = \sqrt{x}\)</span>. The derivative of <span class="math inline">\(f\)</span> is <span class="math inline">\(f'(x) = 2x\)</span> and the derivative <span class="math display">\[\bigl(f^{-1}\bigr)'(x) = \frac{1}{f'(f^{-1}(x))} = \frac{1}{2(\sqrt{x})}.\]</span></p>
<p><strong>Example </strong> Compute a derivative when the independent variable is in the exponent. To compute the derivative of <span class="math inline">\(f: x \to 2^x\)</span>, we need to re-express the function in terms fo the natural base <span class="math inline">\(e\)</span>. To do that, we use to properties of the natural logarithm</p>
<ul>
<li><p>For any positive expression <span class="math inline">\(y\)</span>, <span class="math inline">\(y = e^{ \ln (y) }\)</span> (<span class="math inline">\(\ln\)</span> is the inverse of the exponential function).</p></li>
<li><p><span class="math inline">\(\ln (a^b) = b \ln (a).\)</span></p></li>
</ul>
<p>Then <span class="math inline">\(2^x = e^{ \ln (2^x) } = e^{ x \ln (2) }\)</span>. The derivative is <span class="math inline">\(\ln (2) e^{ x \ln (2) }\)</span>. Re-writing in term of base 2, we obtain <span class="math inline">\(f'(x) = \ln (2) 2^x\)</span>.</p>
<table>
<thead>
<tr class="header">
<th style="text-align: left;">Function</th>
<th style="text-align: left;">Derivative</th>
<th style="text-align: left;">Note</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: left;"><span class="math inline">\(x^a\)</span></td>
<td style="text-align: left;"><span class="math inline">\(ax^{a-1}\)</span></td>
<td style="text-align: left;"><span class="math inline">\(a \in \mathbb{R}\)</span></td>
</tr>
<tr class="even">
<td style="text-align: left;"><span class="math inline">\(\frac{1}{x}\)</span></td>
<td style="text-align: left;"><span class="math inline">\(\frac{-1}{x^2}\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="odd">
<td style="text-align: left;"><span class="math inline">\(x^{\frac 12}\)</span></td>
<td style="text-align: left;"><span class="math inline">\(\frac{1}{2x^{\frac 12}}\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="even">
<td style="text-align: left;"><span class="math inline">\(\ln(x)\)</span></td>
<td style="text-align: left;"><span class="math inline">\(\frac 1x\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="odd">
<td style="text-align: left;"><span class="math inline">\(e^x\)</span></td>
<td style="text-align: left;"><span class="math inline">\(e^x\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="even">
<td style="text-align: left;"><span class="math inline">\(\cosh(x)\)</span></td>
<td style="text-align: left;"><span class="math inline">\(\sinh(x)\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="odd">
<td style="text-align: left;"><span class="math inline">\(\sinh(x)\)</span></td>
<td style="text-align: left;"><span class="math inline">\(\cosh(x)\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="even">
<td style="text-align: left;"><span class="math inline">\(\cos(x)\)</span></td>
<td style="text-align: left;"><span class="math inline">\(-\sin(x)\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="odd">
<td style="text-align: left;"><span class="math inline">\(\sin(x)\)</span></td>
<td style="text-align: left;"><span class="math inline">\(\cos(x)\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="even">
<td style="text-align: left;"><span class="math inline">\(\tan(x)\)</span></td>
<td style="text-align: left;"><span class="math inline">\(1 + \tan^2(x) = \frac{1}{\cos^2(x)}\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="odd">
<td style="text-align: left;"><span class="math inline">\(\dfrac{u(x)}{v(x)}\)</span></td>
<td style="text-align: left;"><span class="math inline">\(\dfrac{v(x)u'(x) - u(x)v'(x)}{v^2(x)}\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="even">
<td style="text-align: left;"><span class="math inline">\(u(x) v(x)\)</span></td>
<td style="text-align: left;"><span class="math inline">\(u'(x)v(x) + u(x)v'(x)\)</span></td>
<td style="text-align: left;"></td>
</tr>
</tbody>
</table>
<h2 id="exercises-on-derivatives">Exercises on derivatives</h2>
<p><strong>Exercice </strong> Compute the derivatives of the following functions</p>
<ul>
<li><p><span class="math inline">\(f_1 : x \to \sqrt{\cos x}.\)</span></p></li>
<li><p><span class="math inline">\(f_2 : x \to \sin(3x + 2).\)</span></p></li>
<li><p><span class="math inline">\(f_3 : x \to e^{\cos x}.\)</span></p></li>
<li><p><span class="math inline">\(f_4 : x \to \ln\bigl(\sqrt{x} \bigr).\)</span></p></li>
<li><p><span class="math inline">\(f_5 : x \to 2^{\ln x}.\)</span></p></li>
</ul>
<p><strong>Correction</strong> <span class="math inline">\(f_1'(x) = -\frac{\sin x}{2\sqrt{\cos x}}.\)</span> <span class="math inline">\(f_2'(x) = 3 \cos(3x + 2).\)</span> <span class="math inline">\(f_3'(x) = -\sin(x) e^{\cos x}.\)</span> <span class="math inline">\(f_4'(x) = \frac{1}{2x}.\)</span> <span class="math inline">\(f_5'(x) = \frac{\ln 2}{x} 2^{\ln x}.\)</span></p>
<h1 id="taylor-series-and-truncated-expansions">Taylor series and truncated expansions</h1>
<p>If a function <span class="math inline">\(f\)</span> is infinitely differentiable (i.e. the <span class="math inline">\(k\)</span>-th derivative function <span class="math inline">\(f^{(k)}\)</span> is continuous for any integer <span class="math inline">\(k\geq 0\)</span>), the Taylor series of <span class="math inline">\(f\)</span> at point <span class="math inline">\(a\)</span> is the series <span class="math display">\[\begin{aligned}
\label{eq_taylor}
f(a) + \frac{f'(a)}{1!}(x - a) + \frac{f''(a)}{2!} (x-a)^2 + \frac{f'''(a)}{3!} (x - a)^3 + ...\end{aligned}\]</span></p>
<p>A function is <strong>analytic</strong> on an open interval <span class="math inline">\(I\)</span> if and only if its Taylor series converges pointwise to the value of the function. Polynomials, exponential and trigonometric function are analytic over all real points. The square root function is not analytic.</p>
<p>The partial sums of the series (or truncated expansion) can be used to approximate a function, and to evaluate it numerically. The <span class="math inline">\(k\)</span>th-order expansion of a function <span class="math inline">\(f\)</span> is the polynomial <span class="math display">\[\begin{aligned}
f(a) + \frac{f'(a)}{1!}(x - a) + \frac{f''(a)}{2!} (x-a)^2 + ... + \frac{f^{(k)}(a)}{k!} (x - a)^k.\end{aligned}\]</span> Truncated expansions are used in implementations of common mathematical functions in computer programs.</p>
<p><strong>Example </strong> The Taylor series of the sine function at point <span class="math inline">\(a = 0\)</span> is <span class="math display">\[\sum_{n=0}^\infty \frac{1}{n!}\frac{d^n\sin(x)}{dx^n} x^n = x - \frac{x^3}{3!} + \frac{x^5}{5!} - \frac{x^7}{7!} + ...\]</span> The 3rd-order expansion is the cubic polynomial <span class="math display">\[x - \frac{x^3}{3!}.\]</span> How good this cubic polynomial approximation to the original sine function? The error is the remainder of the terms of the Taylor series <span class="math display">\[\begin{aligned}
\Bigl| - \frac{x^7}{7!} + \frac{x^9}{9!} - ... \Bigr|. \end{aligned}\]</span> For <span class="math inline">\(|x| < 1\)</span>, the error is bounded by <span class="math inline">\(|x|^7/7!\)</span>. Given that <span class="math inline">\(7! = 5040\)</span>, the error is less than <span class="math inline">\(1/5040 \approx 0.0002\)</span>. Truncated expansions are never good approximations when <span class="math inline">\(|x|\)</span> becomes large, because polynomials are not bounded, but the approximation can be quite good over small intervals around the point at which the Taylor series is computed.</p>
<figure>
<img src="htmltmp/tikzblocks_19.pdf.png" alt="First, third and fifth order expansion of f(x) = \sin(x)." /><figcaption aria-hidden="true">First, third and fifth order expansion of <span class="math inline">\(f(x) = \sin(x)\)</span>.</figcaption>
</figure>
<p>It is important to note that even if a function has a Taylor series, the series may not converge to the function.</p>
<p><strong>Example </strong> Let the function <span class="math display">\[f(x) = \begin{cases}
0 & x \leq 0 \\
e^{-\frac 1x} & x > 0.
\end{cases}.\]</span> Around 0, all the derivatives exist and are equal to 0. The Taylor series of <span class="math inline">\(f\)</span> at point <span class="math inline">\(x = 0\)</span> is 0, but the function itself is different from the identically zero function.</p>
<h2 id="expansion-of-a-function-of-two-variables">Expansion of a function of two variables</h2>
<p>For functions of two variables <span class="math inline">\(f : \mathbb{R}^2 \to \mathbb{R}\)</span>, the Taylor expansion at point <span class="math inline">\((x_0,y_0)^t \in \mathbb{R}^2\)</span> is <span class="math display">\[\begin{aligned}
f(x_0,y_0) = f(x_0,y_0) & + \frac{\partial f(x_0,y_0)}{\partial x} (x - x_0) + \frac{\partial f(x_0,y_0)}{\partial y} (y - y_0) \\
& + \frac{1}{2!} \frac{\partial^2 f(x_0,y_0)}{\partial x^2} (x - x_0)^2 \\
& + \frac{1}{2!} \frac{\partial^2 f(x_0,y_0)}{\partial y^2} (y - y_0)^2 \\
& + \frac{2}{2!} \frac{\partial^2 f(x_0,y_0)}{\partial x \partial y} (x - x_0) (y - y_0)
+ ...\end{aligned}\]</span></p>
<p><strong>Example </strong> The second-order truncated expansion of the function <span class="math inline">\(f(x,y) = y e^{-x}\)</span> at <span class="math inline">\((0,0)^t\)</span> is <span class="math display">\[y - xy.\]</span></p>
<h2 id="expansion-of-a-function-from-mathbbr2-to-mathbbr2">Expansion of a function from <span class="math inline">\(\mathbb{R}^2 \to \mathbb{R}^2\)</span></h2>
<p>Functions <span class="math inline">\(f: \mathbb{R}^2 \to \mathbb{R}^2\)</span> have the form <span class="math display">\[f(x,y) = \begin{pmatrix} f_1(x,y) \\ f_2(x,y) \end{pmatrix}.\]</span> To compute the Taylor series, we need to compute the Taylor series of each function <span class="math inline">\(f_1\)</span> and <span class="math inline">\(f_2\)</span>. For a first-order expansion at <span class="math inline">\((x_0,y_0)^t\)</span>, we obtain the expansion <span class="math display">\[\begin{aligned}
\begin{pmatrix} f_1(x_0,y_0) \\ f_2(x_0,y_0) \end{pmatrix} +
\begin{pmatrix}
\frac{\partial f_1(x_0,y_0)}{\partial x} (x - x_0) + \frac{\partial f_1(x_0,y_0)}{\partial y} (y - y_0) \\
\frac{\partial f_2(x_0,y_0)}{\partial x} (x - x_0) + \frac{\partial f_2(x_0,y_0)}{\partial y} (y - y_0)
\end{pmatrix}\end{aligned}\]</span> Using vector notation <span class="math display">\[\begin{aligned}
\boldsymbol{f}(\boldsymbol{x}) & = \begin{pmatrix} f_1(x,y) \\ f_2(x,y) \end{pmatrix}, \\
D\boldsymbol{f} & =
\begin{pmatrix}
\frac{\partial f_1}{\partial x} & \frac{\partial f_1}{\partial y} \\
\frac{\partial f_2}{\partial x} & \frac{\partial f_2}{\partial y}
\end{pmatrix},\end{aligned}\]</span> we can write a first-order (or linear) approximation of a function <span class="math inline">\(\boldsymbol{f}\)</span> more compactly with <span class="math display">\[\boldsymbol{f}(\boldsymbol{x}) \approx \boldsymbol{f}(\boldsymbol{a}) + D\boldsymbol{f}(\boldsymbol{a}) (\boldsymbol{x} - \boldsymbol{a}).\]</span> The approximation is valid only in a neighbourhood of the point <span class="math inline">\(\boldsymbol{a}\)</span>.</p>
<h1 id="integrals-and-primitives">Integrals and primitives</h1>
<h2 id="primitives">Primitives</h2>
<p>Let <span class="math inline">\(f:I \to \mathbb{R}\)</span>, (<span class="math inline">\(I \subset \mathbb{R}\)</span>). A <strong>primitive</strong> <span class="math inline">\(F\)</span> of <span class="math inline">\(f\)</span> on <span class="math inline">\(I\)</span> is a differentiable map such that <span class="math inline">\(F'(x) = f(x)\)</span>, <span class="math inline">\(x \in I\)</span>.</p>
<p><strong>Example </strong> If <span class="math inline">\(f(x) = x^2\)</span>, we look for <span class="math inline">\(F(x)\)</span> such that <span class="math inline">\(F'(x) = x^2\)</span>. Take <span class="math inline">\(F(x) = x^3/3\)</span>, then <span class="math inline">\(F'(x) = 3x^2/3 = x^2.\)</span> It also work for <span class="math inline">\(F(x) = x^3/3 + C\)</span>, for any constant <span class="math inline">\(C \in \mathbb{R}\)</span>.</p>
<p>We denote the primitive as <span class="math inline">\(F(x) = \int f(x) dx\)</span>. Primitives are unique up to a constant: if <span class="math inline">\(F_1\)</span> and <span class="math inline">\(F_2\)</span> are primitives of <span class="math inline">\(f\)</span>, then <span class="math inline">\(F_1'(x) = f(x)\)</span> and <span class="math inline">\(F_2'(x) = f(x)\)</span>, which implies that <span class="math inline">\(F_1'(x) - F_2'(x) = 0\)</span>. Therefore the difference between <span class="math inline">\(F_1\)</span> and <span class="math inline">\(F_2\)</span>, <span class="math inline">\(G : x \to F_1(x) - F_2(x)\)</span> has a zero derivative: <span class="math inline">\(G'(x) = F_1'(x) - F_2'(x) = 0\)</span>. This means that <span class="math inline">\(G\)</span> is a constant.</p>
<p>Primitives have linear properties. Let <span class="math inline">\(F\)</span> be a primitive of <span class="math inline">\(f\)</span>, <span class="math inline">\(G\)</span> be a primitive of <span class="math inline">\(g\)</span> and <span class="math inline">\(a,b \in \mathbb{R}\)</span>. Then</p>
<ul>
<li><p><span class="math inline">\(F+G\)</span> is a primitive of <span class="math inline">\(f+g\)</span>, or <span class="math display">\[\int \bigl( f(x) + g(x) \bigr) dx = \int f(x)dx + \int g(x)dx.\]</span></p></li>
<li><p><span class="math inline">\(aF\)</span> is a primitive of <span class="math inline">\(af\)</span>, or <span class="math display">\[\int af(x) dx = a \int f(x) dx.\]</span></p></li>
</ul>
<table>
<thead>
<tr class="header">
<th style="text-align: left;">Function</th>
<th style="text-align: left;">Primitive</th>
<th style="text-align: left;">Note</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td style="text-align: left;"><span class="math inline">\(0\)</span></td>
<td style="text-align: left;"><span class="math inline">\(C\)</span></td>
<td style="text-align: left;"><span class="math inline">\(C \in \mathbb{R}\)</span></td>
</tr>
<tr class="even">
<td style="text-align: left;"><span class="math inline">\(a\)</span></td>
<td style="text-align: left;"><span class="math inline">\(ax + C\)</span></td>
<td style="text-align: left;"><span class="math inline">\(a \in \mathbb{R}\)</span></td>
</tr>
<tr class="odd">
<td style="text-align: left;"><span class="math inline">\(x^a\)</span></td>
<td style="text-align: left;"><span class="math inline">\(\frac{x^{a+1}}{a+1}\)</span></td>
<td style="text-align: left;"><span class="math inline">\(a \neq -1\)</span></td>
</tr>
<tr class="even">
<td style="text-align: left;"><span class="math inline">\(x^{-1} = \frac{1}{x}\)</span></td>
<td style="text-align: left;"><span class="math inline">\(\ln |x| + C\)</span></td>
<td style="text-align: left;">(<span class="math inline">\(a = -1\)</span>), <span class="math inline">\(x \neq 0\)</span></td>
</tr>
<tr class="odd">
<td style="text-align: left;"><span class="math inline">\(e^x\)</span></td>
<td style="text-align: left;"><span class="math inline">\(e^x + C\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="even">
<td style="text-align: left;"><span class="math inline">\(\cos(x)\)</span></td>
<td style="text-align: left;"><span class="math inline">\(\sin(x) + C\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="odd">
<td style="text-align: left;"><span class="math inline">\(\sin(x)\)</span></td>
<td style="text-align: left;"><span class="math inline">\(-\cos(x) + C\)</span></td>
<td style="text-align: left;"></td>
</tr>
<tr class="even">
<td style="text-align: left;"><span class="math inline">\(f'(g(x))g'(x)\)</span></td>
<td style="text-align: left;"><span class="math inline">\(f(g(x)) + C\)</span></td>
<td style="text-align: left;"></td>
</tr>
</tbody>
</table>
<p><strong>Exercice </strong> Compute <span class="math display">\[\begin{aligned}
\int { \frac{1}{\sqrt{3x + 5}} } dx\end{aligned}\]</span></p>
<h2 id="integrals">Integrals</h2>
<p>Let <span class="math inline">\(f:[a,b] \to \mathbb{R}\)</span>, and <span class="math inline">\(F\)</span> a primitive of <span class="math inline">\(f\)</span> on <span class="math inline">\([a,b]\)</span>. Then the <strong>definite integral</strong> is the value <span class="math inline">\(F(b) - F(a)\)</span>, and is denoted <span class="math display">\[\begin{aligned}
\int_a^b { f(x) dx } = \bigl[ F(x) \bigr]_a^b . \end{aligned}\]</span></p>
<p>The most important interpretation of the integral is the <strong>area under the curve</strong> (AUC). If <span class="math inline">\(f(x) \geq 0\)</span>, <span class="math inline">\(x \in [a,b]\)</span>, then <span class="math display">\[\int_a^b { f(x) dx }\]</span> is the area under delimited by <span class="math inline">\(f(x)\)</span>, <span class="math inline">\(x\)</span>-axis and the axes <span class="math inline">\(x = a\)</span> and <span class="math inline">\(x = b\)</span>. Negative values integrate as negative areas.</p>
<div class="center">
<p><img src="htmltmp/tikzblocks_20.pdf.png" alt="image" /> <img src="htmltmp/tikzblocks_21.pdf.png" alt="image" /></p>
</div>
<p>If <span class="math inline">\(f\)</span> is a rate (unit in <span><code>something per time</code></span>), and <span class="math inline">\(x\)</span> is <span><code>time</code></span>, then the integral of <span class="math inline">\(f\)</span> has unit <span><code>something</code></span>. This applies to speed (integral: displacement), production or degradation (integral: concentration), etc.</p>
<p>The integral has the following properties</p>
<ul>
<li><p>Linearity. The integral of a sum is the sum of the integrals. <span class="math display">\[\int_a^b { (f(x) + g(x))dx } = \int_a^b f(x) dx + \int_a^b g(x) dx.\]</span></p></li>
<li><p>Negative intervals. <span class="math display">\[\int_a^b { f(x) dx } = - \int_b^a { f(x) dx }.\]</span></p></li>
<li><p>Midpoint rule. <span class="math display">\[\int_a^b { f(x)dx } = \int_a^c f(x)dx + \int_c^b f(x) dx.\]</span> Notice that this works even if <span class="math inline">\(c\)</span> is not between <span class="math inline">\(a\)</span> and <span class="math inline">\(b\)</span>.</p></li>
<li><p>The expression <span class="math display">\[\int_a^x { f(t) dt } = F(x) - F(a) = F_a(x)\]</span> is a function of <span class="math inline">\(x\)</span>. Therefore, for any integrable function <span class="math inline">\(f\)</span>, we have <span class="math display">\[\Bigl( \int_a^x f(x) dx \Bigl)' = f(x).\]</span> This is the <strong>fundamental theorem of calculus</strong>.</p></li>
<li><p><strong>Integration by parts</strong> <span class="math display">\[\int_a^b f'(x)g(x)dx = f(x)g(x)|_a^b - \int_a^b f(x)g'(x)dx.\]</span> Integration by part is excessively useful for computing integral behond simple functions.</p></li>
</ul>
<p><strong>Exercice </strong> Compute the integral <span class="math display">\[\int_0^1 x e^x dx.\]</span></p>
<p><strong>Correction</strong> The integrand is the product of two functions, so let us try integration by parts. Let <span class="math inline">\(g(x) = x\)</span> and <span class="math inline">\(f'(x) = e^x\)</span>. Then <span class="math inline">\(g'(x) = 1\)</span> and <span class="math inline">\(f(x) = e^x\)</span>. The integral becomes <span class="math display">\[\begin{aligned}
\int_0^1 x e^x dx & = \Bigl. e^x x \Bigr|_0^1 - \int_0^1 e^x dx, \\
& = e^1 - 0 - \Bigl. e^x \Bigr|_0^1, \\
& = e - (e^1 - e^0), \\
& = 0 + 1, \\
& = 1.\end{aligned}\]</span></p>
<p><strong>Exercice </strong> Compute the integral <span class="math display">\[\int_0^\pi \cos x e^x dx.\]</span></p>
<p><strong>Correction</strong> The integrand is the product of two functions, we try integration by parts. Let <span class="math inline">\(g(x) = \cos x\)</span> and <span class="math inline">\(f'(x) = e^x\)</span>. Then <span class="math inline">\(g'(x) = -\sin x\)</span> and <span class="math inline">\(f(x) = e^x\)</span>. The integral becomes <span class="math display">\[\begin{aligned}
\int_0^\pi \cos x e^x dx & = \Bigl. e^x (\cos x) \Bigr|_0^\pi - \int_0^\pi e^x (-\sin x)dx, \\
& = - e^\pi - e^0 - \int_0^\pi e^x (-\sin x)dx, \\
& = - e^\pi - 1 - \int_0^\pi e^x (-\sin x)dx, \\\end{aligned}\]</span> We still have a integral of a product to compute. We apply the integral by part once more, <span class="math display">\[\begin{aligned}
& = - e^\pi - 1 - \bigl( \Bigl. e^x ( - \sin x ) \Bigr|_0^\pi + \int_0^\pi e^x (\cos x)dx \bigr). \\
& = - e^\pi - 1 - \int_0^\pi e^x (\cos x)dx. \\\end{aligned}\]</span> With this new integration, we just came back to our initial integral. We are looping, and further integrations by part will not help us. To break the loop, we use the fact that the initial integral term is present on both side of the equation and solve for it. If <span class="math display">\[\begin{aligned}
I = & \int_0^\pi \cos x e^x dx,\end{aligned}\]</span> then <span class="math display">\[\begin{aligned}
I & = - e^\pi - 1 - I, \\
2 I & = - e^\pi - 1, \\
I & = \frac{- e^\pi - 1}{2}.\end{aligned}\]</span> Et voilà!</p>
<p><strong>Exercice </strong> Compute the integral <span class="math display">\[\int_0^1 { \sinh x e^x dx }.\]</span></p>
<h1 id="differential-equations-in-1d">Differential equations in 1D</h1>
<p>An <strong>ordinary differential equation</strong> (ODE) of <strong>order</strong> <span class="math inline">\(n\)</span> is a relation (i.e. an equation) between a real variable <span class="math inline">\(t \in I \subset \mathbb{R}\)</span>, an unknown function <span class="math inline">\(x: t \to x(t)\)</span> and its derivatives <span class="math inline">\(x',x'',x''', ..., x^{(n)}\)</span> at point <span class="math inline">\(t\)</span> defined by <span class="math display">\[F\bigl(t,x,x',x'',...,x^{(n)}\bigr) = 0,\]</span> where <span class="math inline">\(F\)</span> <em>depends</em> on <span class="math inline">\(x^{(n)}\)</span>. In general, <span class="math inline">\(x(t)\)</span> takes values in <span class="math inline">\(\mathbb{R}^N\)</span>, i.e. it is a vector. We say that the equation is a <strong>scalar differential equation</strong> if <span class="math inline">\(N = 1\)</span>. (The expression <span class="math inline">\(x^{(i)}\)</span> stands for the <span class="math inline">\(i\)</span>-th derivative, not the <span class="math inline">\(i\)</span>-th power.)</p>
<p>The <strong>normal form</strong> of a differential equation of order <span class="math inline">\(n\)</span> is <span class="math display">\[x^{(n)} = f\bigl(t,x,x',x'',...,x^{(n-1)}\bigr).\]</span></p>
<p>A differential equation is <strong>autonomous</strong> if it does not depend on <span class="math inline">\(t\)</span>, i.e. <span class="math inline">\(F\)</span> has the form <span class="math display">\[F\bigl(x,x',x'',...,x^{(n)}\bigr) = 0,\]</span></p>
<p>A <strong>linear differential equation</strong> is a differential equation for which <span class="math inline">\(F\)</span> is a linear function in <span class="math inline">\(x,x',...x^{(n)}\)</span>. It can be expressed as <span class="math display">\[a_n(t) x^{(n)} + a_{n-1}(t)x^{(n-1)} + ... a_1(t)x' + a_0(t)x = g(t),\]</span> where the coefficients <span class="math inline">\(a_j(t)\)</span> may depend on <span class="math inline">\(t\)</span>, and <span class="math inline">\(x\)</span> and its derivatives <span class="math inline">\(x^{(i)}\)</span> appear only as monones of degree 1 (that is, linearly). If all coefficients are constants, including <span class="math inline">\(g\)</span>, the linear differential equation is autonomous.</p>
<p><strong>Exercice </strong> For the following differential equations, give the order <span class="math inline">\(n\)</span>, and determine whether they are autonomous, linear, and whether they are expressed under their normal form.</p>
<ol>
<li><p><span class="math inline">\(x - t + 4 t x' = 0.\)</span></p></li>
<li><p><span class="math inline">\(\bigl( x'' \bigr)^2 - 2 x' t x = 0.\)</span></p></li>
<li><p><span class="math inline">\(x^{(3)} + \sin(x') = -5 x.\)</span></p></li>
<li><p><span class="math inline">\(x^{(4)} - x x' = 0 .\)</span></p></li>
<li><p><span class="math inline">\(3x'' - 4 x' + 6 x = 2.\)</span></p></li>
<li><p><span class="math inline">\(\ln x' + 3 t x = \sqrt{x}.\)</span></p></li>
</ol>
<p><strong>Correction</strong></p>
<ol>
<li><p>linear, non-autonomous, not in normal form</p></li>
<li><p>non-linear, non-automous, not in normal form</p></li>
<li><p>non-linear, autonomous, not in normal form</p></li>
<li><p>non-linear, autonomous, not in normal form</p></li>
<li><p>non-linear, non-autonomous, not in normal form</p></li>
</ol>
<p>A <strong>solution or integral of a differential equation</strong> of order <span class="math inline">\(n\)</span> for <span class="math inline">\(t\)</span> in an interval <span class="math inline">\(I \subset \mathbb{R}\)</span>, is a map <span class="math inline">\(x: I -> \mathbb{R}\)</span> that is <span class="math inline">\(n\)</span> times differentiable for all <span class="math inline">\(t \in I\)</span> and satisfies the differential equation.</p>
<p>A <strong>integral curve (or chronic)</strong> is the set of points <span class="math inline">\((t,x(t))\)</span> for <span class="math inline">\(t \in I\)</span>. If <span class="math inline">\(x(t) \in \mathbb{R}^N\)</span>, the integral curve is in <span class="math inline">\(\mathbb{R}^{N+1}\)</span>. A <strong>trajectory or orbit</strong> is the set of points <span class="math inline">\(x(t)\)</span> for <span class="math inline">\(t \in I\)</span>. This is a set in <span class="math inline">\(\mathbb{R}^N\)</span>. The space that contains the trajectories is called the <strong>phase space</strong>. The set of all trajectories is called the <strong>phase portrait</strong>.</p>
<p>It is always possible to write an differential equation of order <span class="math inline">\(n\)</span> as a differential equation of order 1, by defining extra variables for the higher-order derivatives.</p>
<p><strong>Example </strong> We consider the differential equation <span class="math inline">\(a(t) x'' + b(t) x' + c(t) x = d(t)\)</span>. This is a equation of order 2. To reduce it to order 1, let <span class="math inline">\(z_1 = x\)</span> and <span class="math inline">\(z_2 = x'\)</span>. Then <span class="math inline">\(x'' = z_2'\)</span> and <span class="math inline">\(z_1' = z_2\)</span>. The second-order differential equation can be re-expressed as two first order equations: <span class="math display">\[z_1' = z_2, \quad a(t)z_2' + b(t)z_2 + c(t) z_1 = d(t).\]</span> Often, the system of first order equations can be re-expressed in normal form, by isolating the variables <span class="math inline">\(z_1'\)</span> and <span class="math inline">\(z_2'\)</span>, <span class="math display">\[z_1' = z_2, \text{ and } z_2' = \frac{d(t) - b(t)z_2 - c(t) z}{a(t)}.\]</span> (We assume that <span class="math inline">\(a(t) \neq 0\)</span>.) In general, for the differential equation of order <span class="math inline">\(n\)</span> <span class="math inline">\(F(t,x,x',...,x^{(n)}) = 0,\)</span> with <span class="math inline">\(x : I \to \mathbb{R}^m\)</span>, we make a change of variables: <span class="math inline">\(z_1 = x, z_2 = x', ...,
z_i = x^{(i-1)}\)</span> until <span class="math inline">\(z_n = x^{(n-1)}\)</span>. Each variable <span class="math inline">\(z_i(t) \in \mathbb{R}^m\)</span>, so the new vector <span class="math inline">\(z = \bigl( z_1, z_2, ..., z_n \bigr)^t\)</span> is in <span class="math inline">\(\mathbb{R}^{mn}\)</span>. With the change of variables, the differential equation now reads <span class="math display">\[\begin{aligned}
z_1' & = z_2, \\
z_2' & = z_3, \\
& ..., \\
z_i' & = z_{i+1}, \\
& ..., \\
F(t,&z_1,z_2,...,z_n,z_n') = 0. \\\end{aligned}\]</span></p>
<hr />
<p><strong>Tips on Ordinary differential equations</strong></p>
<ul>
<li><p>The most frequently used differential equations are order 1, and they usually are represented in their normal form: <span class="math inline">\(x' = f(x)\)</span> for autonomous equations, and <span class="math inline">\(x' = f(t,x)\)</span> for non-autonomous equations</p></li>
<li><p>For a scalar, autonomous differential equation <span class="math inline">\(x' = f(x)\)</span> with <span class="math inline">\(x(t) \in \mathbb{R}\)</span>, the trajectories are monotonous: if <span class="math inline">\(x\)</span> is a solution, then <span class="math inline">\(x\)</span> is either increasing, decreasing, or constant.</p></li>
</ul>
<hr />
<h2 id="finding-solutions-of-differential-equations">Finding solutions of differential equations</h2>
<p>We consider a <strong>first order scalar differential equation</strong>, <span class="math display">\[a(t) x' + b(t) x = d(t),\]</span> <span class="math inline">\(t \in I\)</span> and <span class="math inline">\(a(t) \neq 0\)</span> on <span class="math inline">\(I\)</span>, <span class="math inline">\(a(t)\)</span> and <span class="math inline">\(b(t)\)</span> continuous on <span class="math inline">\(I\)</span>. If <span class="math inline">\(d(t) = 0\)</span>, we call the equation homogeneous, and <span class="math display">\[a(t) x' + b(t) x = 0.\]</span></p>
<p><em>First order scalar ODE. Homogeneous case, first method.</em> Write the equation in normal form, <span class="math display">\[x' = - \frac{b(t)}{a(t)} x.\]</span> The solution <span class="math inline">\(x\)</span> is either the constant <span class="math inline">\(x = 0\)</span>, or <span class="math inline">\(x(t) \neq 0\)</span> for all <span class="math inline">\(t\in I\)</span>. We know that because of uniqueness of solutions, which implies that trajectories cannot cross. If <span class="math inline">\(x = 0\)</span> we are done, so we can assume that <span class="math inline">\(x(t) \neq 0\)</span>. Dividing the equation by <span class="math inline">\(x\)</span> <span class="math display">\[\frac{x'}{x} = -\frac{b(t)}{a(t)}.\]</span> The terms on both sides are functions of <span class="math inline">\(t\)</span>. We can compute their primitives <span class="math display">\[\int {\frac{x'}{x} dt } = - \int { \frac{b(t)}{a(t)} dt}.\]</span> The integrand of the left-hand side is <span class="math inline">\(x'/x\)</span>. This is a very common form called <strong>log-derivative</strong> and admits the primitve <span class="math inline">\(\ln |x|\)</span>. The right-hand side does not necessarily have a close form, and we leave it as it is. With the integration constant, we obtain implicit solution for <span class="math inline">\(x\)</span> <span class="math display">\[\ln |x| = - \int { \frac{b(t)}{a(t)} dt } + K.\]</span> We would like an explicit solution <span class="math inline">\(x\)</span>, <span class="math display">\[\begin{aligned}
|x| & = e^{- \int { \frac{b(t)}{a(t)} dt } + K}.\end{aligned}\]</span> Therefore, <span class="math display">\[\begin{aligned}
x & = \pm e^K e^{- \int { \frac{b(t)}{a(t)} dt }}.\end{aligned}\]</span> Notice that <span class="math inline">\(t\)</span> is a variable of integration, and does not exist outside the integral, and can be replaced by any other variable. To have <span class="math inline">\(x\)</span> as as function of <span class="math inline">\(t\)</span>, we must define the bounds of the integral. When the domain of definition of <span class="math inline">\(t\)</span> is the interval <span class="math inline">\(I = [t_0,t_1], t_0 < t_1,\)</span> the solution <span class="math inline">\(x(t)\)</span> is obtained by integrating from <span class="math inline">\(t_0\)</span> to <span class="math inline">\(t\)</span>, <span class="math display">\[x = \pm e^K e^{- \int_{t_0}^t { \frac{b(u)}{a(u)} du }},\]</span> where we have replaced the variable of integration by <span class="math inline">\(u\)</span>. Then the definite integral is a function of <span class="math inline">\(t\)</span>. The constant <span class="math inline">\(K\)</span> is determined by the <strong>initial condition</strong> on <span class="math inline">\(x\)</span> at <span class="math inline">\(t_0\)</span>, <span class="math inline">\(x(t_0) = x_0 \in \mathbb{R},\)</span> <span class="math display">\[x(t_0) = \pm e^K e^{- \int_{t_0}^{t_0} { \frac{b(u)}{a(u)} du }} = \pm e^K = x_0.\]</span> The constant <span class="math inline">\(K = \mathrm{sign} \, x_0 \ln x_0.\)</span> The fixed bound at <span class="math inline">\(t_0\)</span> is arbitrary, we could have chosen any other time of reference. However, it is very common to consider differential equations for which we know the value of the solution at the initial time <span class="math inline">\(t_0\)</span>. The problem of solving a differential equation with a given initial condition is called an <strong>initial value problem</strong> (IVP) or a <strong>Cauchy problem</strong>.</p>
<p><em>First order scalar ODE. Homogeneous case, second method.</em> Assume that <span class="math inline">\(a(t) \neq 0\)</span>, and write the equation as <span class="math display">\[x' + \frac{b(t)}{a(t)} x = 0.\]</span> Multiply the equation by the term <span class="math inline">\(e^{\int { \frac{b(t)}{a(t)} dt }}\)</span>, <span class="math display">\[x' e^{\int { \frac{b(t)}{a(t)} dt }} + \frac{b(t)}{a(t)} x e^{\int { \frac{b(t)}{a(t)} dt }} = 0.\]</span> Notice that the first term has the form <span class="math inline">\(x' f\)</span> and the second term the form <span class="math inline">\(x f'\)</span>, with <span class="math display">\[f = e^{\int { \frac{b(t)}{a(t)} dt }}.\]</span> The left-hand-side of the resulting equation, <span class="math inline">\(x' f + x f'\)</span>, is the derivative of the product <span class="math inline">\(xf\)</span>, so the differential equation can be integrated, <span class="math display">\[\begin{aligned}
x e^{\int { \frac{b(t)}{a(t)} dt }} & = K, \\
x & = K e^{ - \int { \frac{b(t)}{a(t)} dt }}.\end{aligned}\]</span> The constant <span class="math inline">\(K\)</span> is determined by a condition set on the solution, as in the first method.</p>
<p><strong>Exercice </strong> Solve the equation <span class="math display">\[2x' + 6x = 0, \quad x(0) = 1.\]</span></p>
<p><strong>Correction</strong> The initial condition <span class="math inline">\(x(0) = 1\)</span> tells us to start the integration at <span class="math inline">\(t = 0\)</span>. Using the second method with <span class="math inline">\(a(t) = 2\)</span> and <span class="math inline">\(b(t) = 6\)</span>, we have <span class="math display">\[x(t) = K e^{- \int_0^t { - \frac 62 du } } = K e^{ - \frac 62 t }.\]</span> The constant <span class="math inline">\(K\)</span> is determined by the initial condition, <span class="math display">\[x(0) = K e^0 = K = 1.\]</span> The complete solution is <span class="math display">\[x(t) = e^{- \frac 62 t}.\]</span></p>
<p><strong>Exercice </strong> Solve the equation <span class="math display">\[x' + \frac 1t x = 0, \quad x(1) = 1.\]</span></p>
<p><strong>Correction</strong> The initial condition <span class="math inline">\(x(1) = 1\)</span> tells us to start the integration at <span class="math inline">\(t = 1\)</span>. Using the second method with <span class="math inline">\(a(t) = 1\)</span> and <span class="math inline">\(b(t) = 1/t\)</span>, we have <span class="math display">\[x(t) = K e^{- \int_1^t { - \frac 1t du } } = K e^{ - \ln u |_1^t } = K e^{ - \ln t}.\]</span> The term <span class="math inline">\(e^{ - \ln t} = e^{\ln t^{-1}} = t^{-1}.\)</span> The solution simplifies to <span class="math display">\[x(t) = \frac Kt.\]</span> The constant <span class="math inline">\(K\)</span> is found with the initial condition <span class="math inline">\(x(1) = K = 1,\)</span> for a complete solution <span class="math inline">\(x(t) = \frac 1t.\)</span></p>
<p><em>First order scalar ODE. Heterogenous case.</em> We now consider the more general differential equation <span class="math display">\[a(t) x' + b(t) x = d(t).\]</span> Using the strategy from the second method for the homogeneous case above, we divide the equation by <span class="math inline">\(a(t)\)</span>, again assuming that <span class="math inline">\(a(t) \neq 0\)</span>, and then multiply by <span class="math inline">\(e^{ \int { \frac{b(t)}{a(t)} dt} }.\)</span> The resulting equation now has a non-zero right-hand-side, <span class="math display">\[x'e^{ \int { \frac{b(t)}{a(t)} dt} } + \frac{b(t)}{a(t)} x e^{ \int { \frac{b(t)}{a(t)} dt} } = \frac{d(t)}{a(t)} e^{ \int { \frac{b(t)}{a(t)} dt} }.\]</span> Nevertheless, the left-hand-side is still of the form <span class="math inline">\(x' f + x f'\)</span>, and the right-hand-side only depends on <span class="math inline">\(t\)</span>. By integrating both sides, we obtain <span class="math display">\[x e^{ \int { \frac{b(t)}{a(t)} dt} } = \int { \frac{d(t)}{a(t)} e^{ \int { \frac{b(t)}{a(t)} dt}} dt } + K.\]</span> The solution for <span class="math inline">\(x\)</span> is then <span class="math display">\[x = \Bigl( \int { \frac{d(t)}{a(t)} e^{ \int { \frac{b(t)}{a(t)} dt}} dt } + K \Bigr) e^{ - \int { \frac{b(t)}{a(t)} dt} }.\]</span></p>
<p><strong>Example </strong> We consider the differential equation <span class="math display">\[\begin{aligned}
x' + 3 x = 1+\sin(t), \quad x(0) = x_0 > 0. \end{aligned}\]</span> The coefficients <span class="math inline">\(a(t) = 1, b(t) = 3, d(t) = 1 + \sin(t).\)</span> The term <span class="math inline">\(e^{ \int { \frac{b(t)}{a(t)} dt}} = e^{ \int { 3 dt } } = e^{3t}\)</span>. The general solution is <span class="math display">\[\begin{aligned}
x(t) & = \Bigl( \int { (1+\sin(t)) e^{3t} dt } + K \Bigr)e^{ - 3t } , \\
& = \frac 13 + \frac {1}{10} \bigl( \sin(t) - \cos(t) \bigr) + Ke^{-3t}. \end{aligned}\]</span> At <span class="math inline">\(t = 0\)</span>, the equation <span class="math inline">\(x(0) = \frac 13 + \frac{1}{10} (0 - 1) + K = x_0\)</span> solve in <span class="math inline">\(K\)</span> as <span class="math inline">\(K = x_0 - \frac{7}{30}\)</span>. The solution <span class="math inline">\(x(t)\)</span> is <span class="math display">\[x(t) = \frac 13 + \frac {1}{10} \bigl( 3 \sin(t) - \cos(t) \bigr) + \Bigl( x_0 - \frac{7}{30} \Bigr) e^{-3t}.\]</span></p>
<figure>
<img src="htmltmp/tikzblocks_22.pdf.png" alt="Solution of the initial value problem for the heterogenous differential equation x' + 3x = 1 + \sin(x)." /><figcaption aria-hidden="true">Solution of the initial value problem for the heterogenous differential equation <span class="math inline">\(x' + 3x = 1 + \sin(x)\)</span>.</figcaption>
</figure>
<p>We now consider a <strong>nonlinear differential equation</strong> of <strong>Bernoulli</strong> type. Bernoulli equations are of the form <span class="math display">\[\begin{aligned}
x' + P(t) x + Q(t) x^r = 0, \quad t \in I \subset \mathbb{R},\end{aligned}\]</span> with continuous functions <span class="math inline">\(P, Q\)</span>, and <span class="math inline">\(r \in \mathbb{R}\)</span>. There is no general method to solve nonlinear differential equations, but is can be done is particular cases. If <span class="math inline">\(r = 0\)</span> or <span class="math inline">\(r = 1\)</span>, the equation is linear, and we already know how to solve it. Suppose <span class="math inline">\(r\)</span> different from 0 or 1. We will look for positive solutions <span class="math inline">\(x(t) > 0\)</span> on <span class="math inline">\(t \in I\)</span>. Dividing by <span class="math inline">\(x^r\)</span>, we get <span class="math display">\[\begin{aligned}
x' x^{-r} + P(t) x x^{-r} + Q(t) x^r x^{-r} & = 0, \\
x' x^{-r} + P(t) x x^{-r} + Q(t) & = 0.\end{aligned}\]</span> We now set an auxiliary variable <span class="math inline">\(u = x^{1-r}\)</span>, (<span class="math inline">\(x = u^{1/(1-r)}\)</span>). Then <span class="math display">\[\begin{aligned}
u' = (1-r)x^{1-r-1}x' = (1-r)x^{-r}x',\end{aligned}\]</span> and, subsituting in the differential equation, <span class="math display">\[\begin{aligned}
\frac{1}{1-r} u' + P(t) u + Q(t) & = 0, \\
u' + (1-r)P(t) u + (1-r) Q(t) & = 0. \end{aligned}\]</span> We know how to solve this equation; this is a linear equation of the form <span class="math display">\[\begin{aligned}
a(t) u' + b(t) u = d(t),\end{aligned}\]</span> with <span class="math inline">\(a(t) = 1, b(t) = (1-r)P(t), d(t) = - (1-r)Q(t)\)</span>.</p>
<p><strong>Example </strong> Verhulst equation (logistic equation) <span class="math display">\[\begin{aligned}
x' = \mu x \Bigl( 1 - \frac xK \Bigr).\end{aligned}\]</span> We first rewrite the equation in Bernoulli form, <span class="math display">\[\begin{aligned}
x' - \mu x + \mu \frac{x^2}{K} = 0,\end{aligned}\]</span> with <span class="math inline">\(P(t) = -1, Q(t) = \mu/K, r = 2\)</span>. The auxiliary equation reads <span class="math display">\[\begin{aligned}
u' + (1-2)(-1) \mu u + (1-2) \frac{\mu}{K} & = 0.
u' + u - \frac{\mu}{K} & = 0. \end{aligned}\]</span> This is a scalar linear equation with <span class="math inline">\(a(t) = 1, b(t) = \mu, d(t) = \mu/K.\)</span>. The general solution is <span class="math display">\[\begin{aligned}
u(t) & = \Bigl( \int { \frac{d(t)}{a(t)}e^{\int b(t)/a(t) dt}dt } + C \Bigr) e^{ - \int { b(t)/a(t) dt } }, \\
\int b(t)/a(t) dt & = \int \mu dt = \mu t, \\
u(t) & = \Bigl( \int { \frac{\mu}{K} e^{\mu t}dt } + C \Bigr) e^{-\mu t}, \\
& = \Bigl( \frac{\mu}{K} \frac{e^{\mu t}}{\mu} + C \Bigr) e^{- \mu t}, \\
& = \frac{1}{K} + C e^{- \mu t}.\end{aligned}\]</span> The initial condition <span class="math inline">\(u(0) = x_0^{1-r} = x_0^{-1}\)</span>, <span class="math display">\[\begin{aligned}
u(0) & = \frac{1}{K} + C = \frac{1}{x_0}, \\
C & = \frac{1}{x_0} - \frac{1}{K}.\end{aligned}\]</span> The solution to the orginal Verhulst equation is <span class="math inline">\(x = u^{-1}\)</span>, <span class="math display">\[\begin{aligned}
x(t) & = \frac{1}{\frac{1}{K} + \bigl( \frac{1}{x_0} - \frac{1}{K} \bigr) e^{-\mu t}} \\
& = \frac{x_0 K}{x_0 + (K- x_0) e^{- \mu t}}.\end{aligned}\]</span></p>
<figure>
<img src="htmltmp/tikzblocks_23.pdf.png" alt="Solution of the initial value problem for the Verhulst equation x' = \mu x (1 - x/K), for x(0) = 0.1 and x(0) = 1.5. Verhulst equation is a type of Bernoulli equation, and can be solved analytically." /><figcaption aria-hidden="true">Solution of the initial value problem for the Verhulst equation <span class="math inline">\(x' = \mu x (1 - x/K)\)</span>, for <span class="math inline">\(x(0) = 0.1\)</span> and <span class="math inline">\(x(0) = 1.5\)</span>. Verhulst equation is a type of Bernoulli equation, and can be solved analytically.</figcaption>
</figure>
<h1 id="complex-numbers">Complex numbers</h1>
<p>A complex number is a number that can be expressed in the form <span class="math inline">\(a + i b\)</span>, where <span class="math inline">\(a\)</span> and <span class="math inline">\(b\)</span> are real numbers, and the symbol <span class="math inline">\(i\)</span> is called <strong>imaginary unit</strong>. The imaginary unit satisfies the equation <span class="math inline">\(i^2 = -1\)</span>. Because no real number satisfies this equation, this number is called <em>imaginary</em>.</p>
<p>For the complex number <span class="math inline">\(z = a + i b\)</span>, <span class="math inline">\(a\)</span> is called the <strong>real part</strong> and <span class="math inline">\(b\)</span> is called the <strong>imaginary part</strong>. The real part of <span class="math inline">\(z\)</span> is denoted <span class="math inline">\(\Re(z)\)</span> (<code>\Re</code> in LaTeX) or just <span class="math inline">\(\mathrm{Re}(z).\)</span> The imaginary part of <span class="math inline">\(z\)</span> denoted <span class="math inline">\(\Im(z)\)</span> (<code>\Im</code> in LaTex) or just <span class="math inline">\(\mathrm{Im}(z)\)</span>. The set of all complex numbers is denoted <span class="math inline">\(\mathbb{C}\)</span> (<code>\mathbb{C}</code> in LaTeX).</p>
<p>We need complex numbers for solving polynomial equations. The fundamental theorem of algebra asserts that a polynomial equation of with real or complex coefficients has complex solutions. These polynomial equations arise when trying to compute the eigenvalues of matrices, something we need to do to solve linear differential equations for instance.</p>
<p>Arithmetic rules that apply on real numbers also apply on complex numbers, by using the rule <span class="math inline">\(i^2 = -1\)</span>: addition, subtraction, multiplication and division are associative, commutative and distributive.</p>
<p>Let <span class="math inline">\(u = a + i b\)</span> and <span class="math inline">\(v = c + i d\)</span> two complex numbers, with real coefficients <span class="math inline">\(a,b,c,d\)</span>. Then</p>
<ul>
<li><p><span class="math inline">\(u + v = a + i b + c + i d = (a+c) + i (b+d)\)</span>.</p></li>
<li><p><span class="math inline">\(uv = (a + i b)(c + i d) = ac + i a d + i b c + i^2 b d = ac - bd + i(ad + bc)\)</span>.</p></li>
<li><p><span class="math inline">\(\frac{1}{v} = \frac{1}{c + i d} = \frac{c - id}{(c - id)(c + id)} = \frac{c - id}{c^2 + d^2} =
\frac{c}{c^2 + d^2} - i \frac{d}{c^2+d^2}\)</span>.</p></li>
<li><p><span class="math inline">\(u = v\)</span> if and only if <span class="math inline">\(a = c\)</span> and <span class="math inline">\(b = d\)</span>.</p>
<p>It follows from the rule on <span class="math inline">\(i\)</span> that</p></li>
<li><p><span class="math inline">\(\frac 1i = -i.\)</span> (Proof: <span class="math inline">\(\frac 1i = \frac{i}{i^2} = \frac{i}{-1} = -i\)</span>.)</p></li>
</ul>
<p>Multiplying by the imaginary unit <span class="math inline">\(i\)</span> is equivalent to a counterclockwise rotation by <span class="math inline">\(\pi/2\)</span> (Figure <a href="#f_complex_rotation" data-reference-type="ref" data-reference="f_complex_rotation">3</a>) <span class="math display">\[ui = (a + ib)i = ia + i^2 b = -b + ia.\]</span></p>
<figure>
<img src="htmltmp/tikzblocks_24.pdf.png" id="f_complex_rotation" alt="Rotation in the complex plane. Multiplication by i is a 90 degree counterclockwise rotation. " /><figcaption aria-hidden="true">Rotation in the complex plane. Multiplication by <span class="math inline">\(i\)</span> is a 90 degree counterclockwise rotation. </figcaption>
</figure>
<p>Let <span class="math inline">\(z = a + ib\)</span> a complex number with real <span class="math inline">\(a\)</span> and <span class="math inline">\(b\)</span>. The <strong>conjugate</strong> of <span class="math inline">\(z\)</span>, denoted <span class="math inline">\(\bar z\)</span>, is <span class="math inline">\(a - ib\)</span>. The conjugate of the conjugate of <span class="math inline">\(z\)</span> is <span class="math inline">\(z\)</span> (<em>reflection</em>, Figure <a href="#f_complex_rotation" data-reference-type="ref" data-reference="f_complex_rotation">3</a>). The <strong>modulus</strong> of <span class="math inline">\(z\)</span>, denoted <span class="math inline">\(|z|\)</span> is <span class="math inline">\(\sqrt{z \bar z}\)</span>. The product <span class="math inline">\(z \bar z = (a+ib)(a-ib)=a^2 + b^2 + i(-ab + ab) = a^2 + b^2\)</span>. The modulus is the complex version of the absolute value, for if <span class="math inline">\(z\)</span> (i.e. <span class="math inline">\(b = 0\)</span>), <span class="math inline">\(|z| = \sqrt{a^2} = |a|\)</span>. It is always a real, positive number, and <span class="math inline">\(|z| = 0\)</span> if and only if <span class="math inline">\(z = 0\)</span>. The modulus also has the property of being the <em>length</em> of the complex number <span class="math inline">\(z\)</span>, if <span class="math inline">\(a\)</span> and <span class="math inline">\(b\)</span> are the sides of a rectangular triangle, then <span class="math inline">\(|z|\)</span> is the hypotenuse.</p>
<p>When simplifying a ratio involving a complex <span class="math inline">\(v\)</span> at the denominator, it is important to convert it to a real number by multiplying the ratio by <span class="math inline">\(\bar v/ \bar v\)</span>. For instance, if <span class="math inline">\(v \neq 0\)</span>, <span class="math display">\[\frac{u}{v} = \frac{u \bar v}{u \bar v} = \frac{u \bar v}{|v|^2}.\]</span> The denominator <span class="math inline">\(|v|^2\)</span> is always a positive real number.</p>
<p>By allowing complex values, nonlinear functions of real numbers like exponentials, logarithms and trigonometric functions can have their domain extended to all real and complex numbers. The most useful extension is the exponential function. Recall that the exponential function <span class="math inline">\(e^x\)</span>, where <span class="math inline">\(e \approx 2.71828\)</span> is Euler’s constant, satisfies the relation <span class="math inline">\(e^{x + y} = e^{x} e^{y}\)</span>. This remains true for complex numbers. The <strong>Euler’s formula</strong> relates the exponential of a imaginary number with trigonometric functions. For a real number <span class="math inline">\(y\)</span>, <span class="math display">\[e^{i y} = \cos(y) + i \sin(y).\]</span> Therefore, for any complex number <span class="math inline">\(z = a + i b\)</span>, the exponential <span class="math display">\[e^{z} = e^{a + ib} = e^a e^{ib} = e^a \bigl( \cos(b) + i \sin(b) \bigr).\]</span></p>
<figure>
<img src="htmltmp/tikzblocks_25.pdf.png" id="f_complex_plane" alt="Complex plane" /><figcaption aria-hidden="true">Complex plane</figcaption>
</figure>
<hr />
<p><strong>Tips on complex numbers</strong></p>
<ul>
<li><p>If <span class="math inline">\(x\)</span> is real, <span class="math inline">\(ix\)</span> is pure imaginary. If <span class="math inline">\(y\)</span> is imaginary, <span class="math inline">\(iy\)</span> is real.</p></li>
<li><p><span class="math inline">\(|i| = 1\)</span>. For any real <span class="math inline">\(\theta\)</span>, <span class="math inline">\(|e^{i \theta}| = 1\)</span>.</p></li>
<li><p><span class="math inline">\(|z_1 z_2| = |z_1| |z_2|\)</span>.</p></li>
<li><p>In particular, <span class="math inline">\(|iz| = |i||z| = |z|\)</span>. (Multpliying by <span class="math inline">\(i\)</span> is a rotation in the complex plane, it does not change the modulus.)</p></li>
</ul>
<hr />
<h2 id="roots-of-a-complex-number">Roots of a complex number</h2>
<p>For complex numbers, the equation <span class="math inline">\(z^n = 1\)</span> has <span class="math inline">\(n\)</span> solutions. They are called the <strong>root of unity</strong>. For <span class="math inline">\(n=2\)</span>, we have the well-known roots <span class="math inline">\(z = \pm 1\)</span>, which are real. What are the roots of <span class="math inline">\(z^3 = 1\)</span>? To find them, we express <span class="math inline">\(z\)</span> in polar coordinates: <span class="math inline">\(z = r e^{i\theta}\)</span>. Then <span class="math display">\[z^3 = (r e^{i\theta})^3 = r^3 e^{i 3\theta} = 1.\]</span> The equation implies that <span class="math inline">\(z\)</span> has modulus 1, so <span class="math inline">\(r = 1\)</span>. The remaining term <span class="math inline">\(e^{i 3\theta} = 1\)</span> implies that <span class="math inline">\(3 \theta\)</span> is a multiple of <span class="math inline">\(2 \pi\)</span> because <span class="math inline">\(e^{i \omega} = 1\)</span> if and only if <span class="math inline">\(\omega = 2 k \pi\)</span>, for some integer <span class="math inline">\(k\)</span>. Therefore <span class="math inline">\(\theta = \frac{2}{3} k \pi\)</span>, for <span class="math inline">\(k = 0, 1, 2, ...\)</span>. How many distinct points do we have? Clearly, <span class="math inline">\(k = 3\)</span> is equivalent to <span class="math inline">\(k = 0\)</span>: <span class="math inline">\(e^{i \frac{2}{3} 3 \pi} = e^{i 2 \pi} = e{i 0}\)</span>. In the same way <span class="math inline">\(k = 4\)</span> is equivalent to <span class="math inline">\(k = 1\)</span>, and so on. Therefore, there are exactly three distinct solutions for <span class="math inline">\(\theta\)</span>: <span class="math inline">\(0, \frac{2}{3} \pi, \frac{4}{3} \pi\)</span> (Figure <a href="#f_roots_unity" data-reference-type="ref" data-reference="f_roots_unity">5</a>).</p>
<figure>
<img src="htmltmp/tikzblocks_26.pdf.png" id="f_roots_unity" alt="The roots of z^3 - 1." /><figcaption aria-hidden="true">The roots of <span class="math inline">\(z^3 - 1\)</span>.</figcaption>
</figure>
<h2 id="exercises-on-complex-numbers">Exercises on complex numbers</h2>
<p><strong>Exercice </strong> Let the complex number <span class="math inline">\(z = 2 + 3 i\)</span>. Compute <span class="math inline">\(\bar z\)</span>, <span class="math inline">\(|z|\)</span>, <span class="math inline">\(|\bar z|\)</span> (compare with <span class="math inline">\(|z|\)</span>), <span class="math inline">\(z^2\)</span>, <span class="math inline">\(\Re(\bar z)\)</span> , <span class="math inline">\(\Im(\bar z)\)</span>, <span class="math inline">\(\frac{z + \bar z}{2}\)</span> , <span class="math inline">\(\frac{z - \bar z}{2}\)</span> , <span class="math inline">\(-z\)</span>, <span class="math inline">\(iz\)</span>.</p>
<p><strong>Correction</strong> <span class="math inline">\(\bar z = 2 - 3i\)</span>, <span class="math inline">\(|z| = \sqrt{2^2 + 3^2} = \sqrt{13}\)</span>, <span class="math inline">\(|\bar z| = \sqrt{2^2 + (-3)^2} = \sqrt{13}\)</span>, we see that <span class="math inline">\(|z| = |\bar z|\)</span>, <span class="math inline">\(\Re(\bar z) = 2\)</span>, <span class="math inline">\(\Im(\bar z) = -3\)</span>, <span class="math inline">\(\frac{z + \bar z}{2} = (2 + 3i + (2 - 3i))/2 = 2\)</span>, <span class="math inline">\(\frac{z - \bar z}{2} = ((2 + 3i - (2 - 3i))/2 = 3i\)</span>, <span class="math inline">\(-z = -2 - 3i\)</span>, <span class="math inline">\(iz = 2i + 3i^2 = -3 + 2i\)</span>.</p>
<p><strong>Exercice </strong> Any complex number can be represented in <strong>polar form</strong>: <span class="math inline">\(z = r ( \cos(\theta) + i \sin(\theta) )\)</span>.</p>
<ul>
<li><p>Show that <span class="math inline">\(|z| = r\)</span></p></li>
<li><p>Show that <span class="math inline">\(z = r e^{i \theta}\)</span></p></li>
<li><p>Conclude that for any complex number <span class="math inline">\(z\)</span>, <span class="math inline">\(|z| = 1\)</span> if and only if <span class="math inline">\(z\)</span> can be expressed as <span class="math inline">\(z = e^{i \theta}\)</span> for a real <span class="math inline">\(\theta\)</span>.</p></li>
</ul>
<p><strong>Correction</strong> The modulus of <span class="math inline">\(z\)</span> is <span class="math inline">\(|z| = \sqrt{r^2 \cos^2 (\theta) + r^2 \sin^2 (\theta)} = \sqrt{r^2} = r.\)</span> From Euler’s formula, we have <span class="math inline">\(\cos(\theta) + i\sin(\theta) = e^{i\theta}\)</span>, so <span class="math inline">\(z = re^{i\theta}.\)</span> Therefore, for any complex number <span class="math inline">\(z = r e^{i\theta}\)</span>, <span class="math inline">\(|z| = 1\)</span> if and only if <span class="math inline">\(r = 1\)</span>.</p>
<p><strong>Exercice </strong> Using Euler’s formula, show that <span class="math inline">\(\cos(a)\cos(b) - \sin(a)\sin(b) = \cos(a+b)\)</span>. <em>(Use the property that <span class="math inline">\(e^{ia + ib} = e^{ia} e^{ib}\)</span> and apply Euler’s Formula)</em>.</p>
<p><strong>Correction</strong> All trigonometric identities can be obtained by applying Euler’s formula. Here we start from <span class="math inline">\(e^{ia + ib} = \cos(a+b) + i \sin(a+b)\)</span>. We only want the real part, <span class="math display">\[\begin{aligned}
\cos(a + b) & = && \frac{e^{ia + ib} + e^{-ia - ib}}{2} \\
& = && \frac{e^{ia} e^{ib} + e^{-ia} e^{-ib}}{2} \\
& = && \frac{(\cos(a) + i \sin(a))(\cos(b)+ i \sin(b)) + (\cos(a) - i \sin(a))(\cos(b) - i \sin(b))}{2} \\
& = && \frac{\cos(a)\cos(b) + i^2 \sin(a)\sin(b) + i \cos(a)\sin(b) + i \cos(b)\sin(a)}{2} \\
& && + \frac{\cos(a)\cos(b) + i^2 \sin(a)\sin(b) - i \cos(a)\sin(b) - i \cos(b)\sin(a)}{2} \end{aligned}\]</span> The mixed cosine-sine terms cancel each other while the other ones add up, resuling in <span class="math display">\[\begin{aligned}
\cos(a + b) = \cos(a)\cos(b) - \sin(a)\sin(b).\end{aligned}\]</span></p>
<p><strong>Exercice </strong> Show Euler’s identity: <span class="math inline">\(e^{i \pi} = -1\)</span>.</p>
<p><strong>Correction</strong> This is a direct application of Euler’s formula: <span class="math inline">\(e^{i \pi} = \cos(\pi) + i \sin(\pi) = -1.\)</span></p>
<p><strong>Exercice </strong> What are the roots of the equation <span class="math inline">\(z^6 = 1\)</span>?</p>
<p><strong>Correction</strong> The roots must satisfy <span class="math inline">\(e^{i 6 \theta} = 1\)</span>. This means that <span class="math inline">\(\theta = \frac{2}{6} k \pi\)</span>, for <span class="math inline">\(k = 0, 1, ..., 5\)</span>. There are six distinct roots.</p>
<p><strong>Exercice </strong> For a complex <span class="math inline">\(z\)</span>, find necessary and sufficient conditions for <span class="math inline">\(e^{z t}\)</span>, <span class="math inline">\(t > 0\)</span>, to converge to 0.</p>
<p><strong>Correction</strong> The exponential converges to zero if and only if <span class="math inline">\(\Re(z) < 0\)</span>. A complex number is close to zero if and only if its modulus is close to zero. Therefore, to show that a quantity converges to zero, it is necessary and sufficient to show that its modulus converges to zero. If <span class="math inline">\(z = a + ib\)</span>, the exponential <span class="math inline">\(e^{z t} = e^{(a + ib)t} = e^{at} e^{ibt}.\)</span> The modulus <span class="math inline">\(|e^{ibt}| = 1\)</span>, so <span class="math inline">\(|e^{zt}| = e^{at}\)</span> (no need for absolute values, the exponential of a real number is always positive). The condition for convergence to zero is therefore a condition on the real part of <span class="math inline">\(z\)</span>: <span class="math inline">\(e^{at} \to 0\)</span> when <span class="math inline">\(t \to \infty\)</span> if and only if <span class="math inline">\(a < 0\)</span>.</p>
<p><strong>Exercice </strong> Let the complex number <span class="math inline">\(z = a + ib\)</span> with real <span class="math inline">\(a\)</span> and <span class="math inline">\(b\)</span>. Compute <span class="math inline">\(\sqrt{z}\)</span> (that is, express <span class="math inline">\(s = \sqrt{z}\)</span> as <span class="math inline">\(s = \alpha + i \beta\)</span>, with real <span class="math inline">\(\alpha\)</span> and <span class="math inline">\(\beta\)</span>)</p>
<div class="center">
<p><img src="htmltmp/tikzblocks_27.pdf.png" alt="image" /></p>
</div>
<p><strong>Correction</strong> The square root of a complex number always exists. Express <span class="math inline">\(z\)</span> in polar form <span class="math inline">\(z = re^{i\theta}\)</span>, <span class="math inline">\(r \geq 0, \theta \in [0, 2\pi]\)</span>. The square root <span class="math inline">\(\sqrt{z} = \sqrt{r}\sqrt{e^{i\theta}} = \sqrt{r} e^{\frac 12 i \theta}\)</span>. Using Euler’s formula, <span class="math inline">\(\sqrt{z} = \sqrt{r} \cos(\theta/2) + i \sqrt{r} \sqrt{\theta/2}\)</span>. That is, the square root is obtained by taking the square root of the modulus <span class="math inline">\(r\)</span>, and dividing the angle (the <strong>argument</strong>) by 2. There is a problem with this solution, because <span class="math inline">\(z\)</span> can also be represented by <span class="math inline">\(re^{i\theta + 2\pi}\)</span>, giving <span class="math inline">\(\sqrt{z} = \sqrt{r} e^{\frac 12 i\theta + \pi}\)</span>, which is equivalent to dividing the angle by two in the other direction. We define the <strong>principal square root</strong> as the solution that makes the smallest change in angle: <span class="math inline">\(\sqrt{z} = \sqrt{r} e^{\frac 12 i\theta}\)</span> if <span class="math inline">\(\theta \in [0,\pi]\)</span>, and <span class="math inline">\(\sqrt{z} = \sqrt{r} e^{\frac 12 i\theta + \pi}\)</span> if <span class="math inline">\(\theta \in (\pi,2\pi]\)</span> To express the solution in terms of the original form of <span class="math inline">\(z = a + i b\)</span>, we express the square root <span class="math inline">\(s = \alpha + i \beta\)</span>. Then <span class="math inline">\(s^2 = \alpha^2 - \beta^2 + 2i \alpha \beta = z = a + ib.\)</span> By identifying the real and imaginary parts, we get two equations: <span class="math inline">\(\alpha^2 - \beta^2 = a\)</span> and <span class="math inline">\(2 i \alpha \beta = b.\)</span> Denoting the modulus of <span class="math inline">\(z\)</span> by <span class="math inline">\(r = \sqrt{a^2 + b^2}\)</span>, we can obtain the solutions <span class="math display">\[\alpha = \frac{1}{\sqrt{2}}\sqrt{a + r}, \quad \beta = \mathrm{sign}(b) \frac{1}{\sqrt{2}} \sqrt{-a + r}.\]</span></p>
<h1 id="matrices-in-dimension-2">Matrices in dimension 2</h1>
<h2 id="eigenvalues-of-a-2-times-2-matrix">Eigenvalues of a <span class="math inline">\(2 \times 2\)</span> matrix</h2>
<p>A <span class="math inline">\(2 \times 2\)</span> matrix <span class="math inline">\(A\)</span> is an array with 2 rows and 2 columns: <span class="math display">\[A = \begin{pmatrix}
a & b \\
c & d
\end{pmatrix}.\]</span> Usually, the <strong>coefficients</strong> <span class="math inline">\(a, b, c, d\)</span> are real numbers. The <strong>identity</strong> matrix is the matrix <span class="math display">\[I = \begin{pmatrix}
1 & 0 \\
0 & 1
\end{pmatrix}.\]</span></p>
<p>The <strong>determinant</strong> of <span class="math inline">\(A\)</span>, denoted <span class="math inline">\(\det A\)</span> or <span class="math inline">\(|A|\)</span> is the number <span class="math inline">\(ad - bc\)</span>. The <strong>trace</strong> of <span class="math inline">\(A\)</span>, denoted <span class="math inline">\(\mathrm{tr}\,A\)</span>, is the sum of the main <strong>diagonal</strong> of <span class="math inline">\(A\)</span>: <span class="math inline">\(a + d\)</span>.</p>
<p>The <strong>characteristic polynomial</strong> of <span class="math inline">\(A\)</span> is the second order polynomial in <span class="math inline">\(\lambda\)</span> obtained by computing the determinant of the matrix <span class="math inline">\(A - \lambda I\)</span> (Figure <a href="#f_characteristic_polynomial" data-reference-type="ref" data-reference="f_characteristic_polynomial">6</a>), <span class="math display">\[\det ( A - \lambda I ) = (a-\lambda)(d-\lambda) - bc = ad - bc - \lambda ( a + d ) + \lambda^2.\]</span> The characteristic polynomial <span class="math inline">\(p_A(\lambda)\)</span> of <span class="math inline">\(A\)</span> can be expressed in terms of its determinant and trace: <span class="math display">\[p_A(\lambda) = \det A - \mathrm{tr}\,A \lambda + \lambda^2.\]</span> The <strong>eigenvalues</strong> of <span class="math inline">\(A\)</span> are the roots of the characteristic polynomial. By the fundamental theorem of algebra, we know that the characteristic polynomial has exactly two roots, counting multiple roots. These roots can be real, or complex. The eigenvalues of <span class="math inline">\(A\)</span> are calculated using the quadratic formula: <span class="math display">\[\lambda_{1,2} = \frac{1}{2} \Bigl( \mathrm{tr}\,A \pm \sqrt{ (\mathrm{tr}\,A)^2 - 4 \det A } \Bigr).\]</span></p>
<figure>
<img src="htmltmp/tikzblocks_28.pdf.png" id="f_characteristic_polynomial" alt="Graph of the characteristic polynomial. (a, green) polynomial with two negative roots, p(\lambda) = 0.08 + 0.8\lambda + \lambda^2. (b, gray) polynomial with two complex roots, p(\lambda) = 0.1 - 0.3\lambda + \lambda^2. (c, red) polynomial with two positive roots, p(\lambda) = 0.05 + 0.5\lambda + \lambda^2. (d, purple) polynomial with a negative and a positive root, p(\lambda) = -0.1 - 0.3\lambda + \lambda^2. " /><figcaption aria-hidden="true">Graph of the characteristic polynomial. (<em>a, green</em>) polynomial with two negative roots, <span class="math inline">\(p(\lambda) = 0.08 + 0.8\lambda + \lambda^2\)</span>. (<em>b, gray</em>) polynomial with two complex roots, <span class="math inline">\(p(\lambda) = 0.1 - 0.3\lambda + \lambda^2\)</span>. (<em>c, red</em>) polynomial with two positive roots, <span class="math inline">\(p(\lambda) = 0.05 + 0.5\lambda + \lambda^2\)</span>. (<em>d, purple</em>) polynomial with a negative and a positive root, <span class="math inline">\(p(\lambda) = -0.1 - 0.3\lambda + \lambda^2\)</span>. </figcaption>
</figure>
<p>From this formula, we can classify the eigenvalues of <span class="math inline">\(A\)</span>. Let <span class="math display">\[\Delta = (\mathrm{tr}\,A)^2 - 4 \det A\]</span> the <strong>discriminant</strong> of the quadratic formula. The two eigenvalues of <span class="math inline">\(A\)</span> are real if and only if <span class="math inline">\(\Delta \geq 0\)</span>, i.e. <span class="math inline">\(\mathrm{tr}\,A)^2 \geq 4 \det A\)</span> Then we have the following properties (Figure <a href="#f_eigenvalues" data-reference-type="ref" data-reference="f_eigenvalues">7</a>):</p>
<ol>
<li><p><span class="math inline">\(\Delta < 0\)</span>, complex eigenvalues</p>
<ul>
<li><p>The two eigenvalues are complex conjugate: <span class="math inline">\(\lambda_1 = \bar \lambda_2\)</span></p></li>
<li><p>Their real part <span class="math inline">\(\Re(\lambda) = \frac{1}{2} \mathrm{tr}\,A\)</span>.</p></li>
</ul></li>
<li><p><span class="math inline">\(\Delta = 0\)</span>, there is a single root of multiplicity 2: <span class="math inline">\(\lambda = \frac{1}{2} \mathrm{tr}\,A\)</span>.</p></li>
<li><p><span class="math inline">\(\Delta > 0, \det A > 0\)</span>, real, distinct eigenvalues of the same sign.</p>
<ul>
<li><p><span class="math inline">\(\mathrm{tr}\,A > 0\)</span> and <span class="math inline">\(\det A > 0\)</span>. Then <span class="math inline">\(\lambda_{1,2}\)</span> are distinct and positive.</p></li>
<li><p><span class="math inline">\(\mathrm{tr}\,A < 0\)</span> and <span class="math inline">\(\det A > 0\)</span>. Then <span class="math inline">\(\lambda_{1,2}\)</span> are distinct and negative.</p></li>
</ul></li>
<li><p><span class="math inline">\(\det A < 0\)</span>, real distinct eigenvalues of opposite sign.</p>
<ul>
<li><p><span class="math inline">\(\lambda_1 < 0 < \lambda_2\)</span>.</p></li>
</ul></li>
<li><p><span class="math inline">\(\det A = 0\)</span> one of the eigenvalue is zero, the other eigenvalue is <span class="math inline">\(\mathrm{tr}\,A\)</span>.</p></li>
</ol>
<figure>
<img src="htmltmp/tikzblocks_29.pdf.png" id="f_eigenvalues" alt="Properties of the eigenvalues of a 2 \times 2 matrix A with respect to \det A and \mathrm{tr}\,A." /><figcaption aria-hidden="true">Properties of the eigenvalues of a <span class="math inline">\(2 \times 2\)</span> matrix <span class="math inline">\(A\)</span> with respect to <span class="math inline">\(\det A\)</span> and <span class="math inline">\(\mathrm{tr}\,A\)</span>.</figcaption>
</figure>
<h3 id="exercises-on-eigenvalues">Exercises on eigenvalues</h3>
<p><strong>Exercice </strong> Properties of the eigenvalues of <span class="math inline">\(2 \times 2\)</span> matrices. For each <span class="math inline">\(2 \times 2\)</span> matrix, compute the determinant, the trace, and the discriminant, and determine whether the eigenvalues are real, complex, distinct, and the sign (negative, positive, or zero) of the real parts.</p>
<p><span class="math display">\[A_1 = \begin{pmatrix}
0 & -1 \\
1 & 0
\end{pmatrix}, \quad
A_2 = \begin{pmatrix}
-2 & 1 \\
1 & -2
\end{pmatrix}, \quad
A_3 = \begin{pmatrix}
1 & -2 \\
0 & 1
\end{pmatrix}, \quad
A_4 = \begin{pmatrix}
-1 & 2 \\
1/2& 2
\end{pmatrix}.\]</span></p>
<h2 id="matrix-vector-operations">Matrix-vector operations </h2>
<p>A matrix defines a linear transformation between vector spaces. Given a vector <span class="math inline">\(x\)</span>, the product <span class="math inline">\(Ax\)</span> is vector composed of linear combinations of the coefficients of <span class="math inline">\(x\)</span>. For a matrix <span class="math inline">\(2 \times 2\)</span>, the vector <span class="math inline">\(x\)</span> must be a vector of size 2, and the product <span class="math inline">\(Ax\)</span> is a vector of size two. If <span class="math inline">\(x = (x_1, x2)^t\)</span> (the <span class="math inline">\({}^t\)</span> stands for the transpose, because <span class="math inline">\(x\)</span> must be a column vector), and <span class="math inline">\(A = [ a_{ij} ]_{i=1,2, \, j=1,2}\)</span>, then <span class="math display">\[Ax =
\begin{pmatrix}
a_{11} & a_{12} \\
a_{21} & a_{22}
\end{pmatrix}
\begin{pmatrix}
x_{1} \\
x_{2}
\end{pmatrix}
=
\begin{pmatrix}
a_{11} x_{1} + a_{12} x_2 \\
a_{21} x_{1} + a_{22} x_2
\end{pmatrix}.\]</span> Successive linear transformations can be accomplished by applying several matrices. Given two matrices <span class="math inline">\(A, B\)</span>, the matrix product <span class="math inline">\(C = AB\)</span> is also a matrix. The matrix <span class="math inline">\(C\)</span> is the linear transformation that first applies <span class="math inline">\(B\)</span>, then <span class="math inline">\(A\)</span>. Matrix product is <em>not</em> commutative is general: <span class="math inline">\(AB \neq BA\)</span>. <em>(If <span class="math inline">\(B\)</span> means ’put on socks’ and <span class="math inline">\(A\)</span> means ’put on shoes’, then <span class="math inline">\(BA\)</span> does not have the expected result.)</em> The product of two matrices <span class="math inline">\(A = [ a_{ij} ]_{i=1,2, \, j=1,2}\)</span> and <span class="math inline">\(B = [ b_{ij} ]_{i=1,2, \, j=1,2}\)</span> is <span class="math display">\[AB =
\begin{pmatrix}
a_{11} & a_{12} \\
a_{21} & a_{22}
\end{pmatrix}
\begin{pmatrix}
b_{11} & b_{12} \\
b_{21} & b_{22}
\end{pmatrix}
=
\begin{pmatrix}
a_{11} b_{11} + a_{12}b_{21} & a_{11}b_{12} + a_{12}b_{22} \\
a_{21} b_{11} + a_{22}b_{21} & a_{21}b_{12} + a_{22}b_{22}
\end{pmatrix}.\]</span> The <strong>sum of two matrices</strong> <span class="math inline">\(A+B\)</span> is performed element-wise: <span class="math inline">\(A+B = [a_{ij} + b_{ij}]_{i=1,2, \, j=1,2}\)</span>. The <strong>sum of two vectors</strong> is defined similarly. Addition is commutative. Matrix operations are associative and distributive. <span class="math display">\[\begin{aligned}
A + B & = B + A, \\
A(B + C) & = AB + BC, \\
A(BC) & = (AB)C. \end{aligned}\]</span> Matrices and vectors can be multiplied by a scalar value (real or complex). <strong>Multiplication by a scalar</strong> is associative, distributive, and commutative. The result of the multiplication by a scalar is to multiply each coefficient of the matrix or vector by the scalar. For example, if <span class="math inline">\(\lambda, \mu\)</span> are scalars, <span class="math display">\[\begin{aligned}
\lambda A & = A (\lambda I) = A \lambda, \\
\lambda ( A + B ) & = \lambda A + \lambda B, \\
(\lambda A) B & = \lambda (AB) = A (\lambda B), \\
(\mu + \lambda) A & = \mu A + \lambda A, \\
\mu (\lambda A) & = (\mu \lambda) A, ...\end{aligned}\]</span> The product between two column vectors is not defined, because the sizes do not match. However, we can define the <strong>scalar product</strong> between two column vectors <span class="math inline">\(x, y\)</span> in the same way matrix product is defined: <span class="math display">\[x^ty \equiv x_1 y_1 + x_2 y_2.\]</span></p>
<p>If the vectors are complex-valued, we need also to conjugate the transposed vector <span class="math inline">\(x^t\)</span>. The conjugate-transpose is called the <strong>adjoint</strong> and is denoted <span class="math inline">\({}^*\)</span>. Thus, if <span class="math inline">\(x\)</span> is complex-valued, the adjoint <span class="math inline">\(x^*\)</span> is the row vector <span class="math inline">\((\bar x_1, \bar x_2)\)</span>. The scalar product for complex-valued vectors is denoted <span class="math inline">\(x^*y\)</span>. Since this notation also works for real-valued vector, we will used most of the time.</p>
<p>Two vectors are <strong>orthogonal</strong> if their scalar product is 0. In the plane, this means that they are oriented at 90 degree apart. Orthogonal vectors are super important because they can be used to build orthogonal bases that are necessary for solving all sorts of <em>linear problems</em>.</p>
<h3 id="exercises-on-matrix-vector-and-matrix-matrix-operations">Exercises on Matrix-vector and matrix-matrix operations</h3>
<p><strong>Exercice </strong> Compute matrix-vector product <span class="math display">\[\begin{pmatrix}
0 & -1 \\
1 & 0
\end{pmatrix}
\begin{pmatrix}
x_1 \\ x_2
\end{pmatrix}.\]</span> What is the transformation given by this matrix.</p>
<p><strong>Correction</strong> <span class="math display">\[\begin{pmatrix}
0 & -1 \\
1 & 0
\end{pmatrix}
\begin{pmatrix}
x_1 \\ x_2
\end{pmatrix}
=
\begin{pmatrix}
-x_2 \\ x_1
\end{pmatrix}.\]</span> The transformation is a 90 degree counterclockwise rotation.</p>
<p><strong>Exercice </strong> Compute the matrix-matrix product <span class="math display">\[\begin{pmatrix}
0 & -1 \\
1 & 0
\end{pmatrix}
\begin{pmatrix}
1 & 0 \\
0 & -1
\end{pmatrix}.\]</span> Can you tell what transformation this is?</p>
<p><strong>Correction</strong> <span class="math display">\[\begin{pmatrix}
0 & -1 \\
1 & 0
\end{pmatrix}
\begin{pmatrix}
1 & 0 \\
0 & -1
\end{pmatrix} =
\begin{pmatrix}
0 & 1 \\
1 & 0
\end{pmatrix}.\]</span> This matrix exchanges the coordinates of a vector, this is a reflection through the axis <span class="math inline">\(x = y\)</span>.</p>
<p><strong>Exercice </strong> Now compute the product of the same matrices, but in the inverse order <span class="math display">\[\begin{pmatrix}
1 & 0 \\
0 & -1
\end{pmatrix}
\begin{pmatrix}
0 & -1 \\
1 & 0
\end{pmatrix}.\]</span> Compare with the solution found in the previous exercise. What is this transformation?</p>
<p><strong>Correction</strong> <span class="math display">\[\begin{pmatrix}
1 & 0 \\
0 & -1
\end{pmatrix}
\begin{pmatrix}
0 & -1 \\
1 & 0
\end{pmatrix} =
\begin{pmatrix}
0 & -1 \\
-1 & 0
\end{pmatrix}.\]</span> The pruduct is not the same, the matrices do not commute. The transformation is now a reflection through <span class="math inline">\(y = -x\)</span>.</p>
<p><strong>Exercice </strong> Find the matrix that takes a vector <span class="math inline">\(x = (x_1,x_2)^t\)</span> and returns <span class="math inline">\((a x_1, b x_2)^t\)</span>.</p>
<p><strong>Correction</strong> The matrix is <span class="math display">\[\begin{pmatrix}
a & 0 \\
0 & b
\end{pmatrix}.\]</span></p>
<p><strong>Exercice </strong> Find the matrix that takes a vector <span class="math inline">\(x = (x_1,x_2)^t\)</span> and returns <span class="math inline">\((x_2, x_1)^t\)</span>.</p>
<p><strong>Correction</strong> The matrix is <span class="math display">\[\begin{pmatrix}
0 & 1 \\
1 & 0
\end{pmatrix}.\]</span></p>
<p><strong>Exercice </strong> Find the matrix that takes a vector <span class="math inline">\(x = (x_1,x_2)^t\)</span> and returns <span class="math inline">\((x_2, 0)^t\)</span>.</p>
<p><strong>Correction</strong> The matrix is <span class="math display">\[\begin{pmatrix}
0 & 1 \\
0 & 0
\end{pmatrix}.\]</span></p>
<p><strong>Exercice </strong> Compute the successive powers <span class="math inline">\(A, A^2, A^3, ...\)</span>, for a diagonal matrix <span class="math inline">\(A\)</span>: <span class="math display">\[A =\begin{pmatrix}
a & 0 \\
0 & b
\end{pmatrix}\]</span></p>
<p><strong>Correction</strong> The power of a diagonal matrix is a diagonal matrix <span class="math display">\[A^k =
\begin{pmatrix}
a^k & 0 \\
0 & b^k
\end{pmatrix}.\]</span></p>
<p><strong>Exercice </strong> Compute the scalar product <span class="math inline">\(x^*y\)</span> between <span class="math inline">\(x = (1 + 2i, 1 - i)^t\)</span> and <span class="math inline">\(y = (0.5 - i, -0.5)^t\)</span>.</p>
<p><strong>Correction</strong> The scalar product is <span class="math display">\[\begin{aligned}
x^*y & = (1 - 2i, 1 + i) (0.5 - i, -0.5)^t \\
& = (1 - 2i)(0.5 - i) + (1 + i)(-0.5) \\
& = 0.5 + 2i^2 - 2(0.5)i - i -0.5 - 0.5i \\
& = (0.5 - 0.5 + 2i^2) + (-2(0.5) - 1 - 0.5)i \\
& = -2 - 2.5i.\end{aligned}\]</span></p>
<p><strong>Exercice </strong> Now compute the scalar product <span class="math inline">\(y^*x\)</span> and compare with the result with the previous exercise.</p>
<p><strong>Correction</strong> The scalar product is <span class="math display">\[\begin{aligned}
y^*x & = (0.5 + i, -0.5) (1 + 2i, 1 - i)^t \\
& = (0.5 + i)(1 + 2i) + (-0.5)(1 - i) \\
& = 0.5 + 2i^2 + i + 2(0.5)i - 0.5 + 0.5i \\
& = -2 + 2.5i\end{aligned}\]</span> This is the conjugate: <span class="math inline">\(x^*y = (y^*x)^*.\)</span></p>
<p><strong>Exercice </strong> Compute the scalar product between <span class="math inline">\(z = (z_1, z_2)^t\)</span> and itself, if <span class="math inline">\(z\)</span> is a complex-valued vector. What can you say about the result?</p>
<p><strong>Correction</strong> The scalar product <span class="math inline">\(z^*z = (\bar z_1, \bar z_2) (z_1, z_2)^t = \bar z_1 z_1 + \bar z_2 z_2 = |z_1|^2 + |z_2|^2.\)</span> The scalar product is the square of the norm of the vector <span class="math inline">\(z\)</span>.</p>
<hr />
<p><strong>Tips on eigenvalues</strong> Some matrices have special shapes that make it easier to compute the determinant, and the eigenvalues. These are called eigenvalue-revealing shapes.</p>
<ul>
<li><p>Diagonal matrices have their eigenvalues on the diagonal.</p></li>
<li><p><strong>Triangular matrices</strong>, i.e. matrices that have zeros above (lower-triangular matrix) or below (upper-triangular matrix) the main diagonal have also their eigenvalues on the diagonal.</p></li>
<li><p>A matrix with a row or a column of zeros has its determinant equal to zero. This implies that one of its eigenvalues is 0.</p></li>
</ul>
<hr />
<h1 id="eigenvalue-decomposition">Eigenvalue decomposition</h1>
<p>In many applications, it is useful to decompose a matrix into a form that makes it easier to operate complex operations on. For instance, we might want to compute the powers of a matrix <span class="math inline">\(A\)</span>: <span class="math inline">\(A^2\)</span>, <span class="math inline">\(A^3\)</span>, <span class="math inline">\(A^4\)</span>. Multiplying matrices are computationally intensive, especially when the size of the matrix becomes large. The <strong>power of a matrix</strong> is <span class="math inline">\(A^k = AA...A\)</span>, <span class="math inline">\(k\)</span> times. The zeroth power is the identity matrix: <span class="math inline">\(A^0 = I\)</span>.</p>
<p>The <strong>inverse</strong> of a matrix <span class="math inline">\(A\)</span>, denoted by <span class="math inline">\(A^{-1}\)</span> is the unique matrix such that <span class="math inline">\(AA^{-1} A{^-1}A = I\)</span>. The notation is self-consistent with the positive powers of <span class="math inline">\(A\)</span>. The inverse does not always exist. A matrix is <strong>invertible</strong> if and only if its determinant is not 0. If <span class="math inline">\(A\)</span> and <span class="math inline">\(B\)</span> are invertible, then <span class="math inline">\(AB\)</span> is invertible, and <span class="math inline">\((AB)^{-1} = B^{-1}A^{-1}\)</span>.</p>
<p>The <strong>eigenvalue decomposition</strong> is a decomposition of the form <span class="math inline">\(A = X D X^{-1}\)</span>, where <span class="math inline">\(D\)</span> is a diagonal matrix, and <span class="math inline">\(X\)</span> is an invertible matrix. If there exists such a decomposition for <span class="math inline">\(A\)</span>, then computing powers of <span class="math inline">\(A\)</span> becomes easy: <span class="math display">\[\begin{aligned}
A^k & = (XDX^{-1})^k = XDX^{-1} \, XDX^{-1} \, ... XDX^{-1}, \\
& = XD(X^{-1}X)D(X^{-1}X) D ... (X^{-1}X) DX^{-1}, \\
& = XD^kX^{-1}.\end{aligned}\]</span></p>
<p>The eigenvalue decomposition does not always exists, because it is not always possible to find an invertible matrix <span class="math inline">\(X\)</span>. When it exists, though, the columns of the matrix <span class="math inline">\(X\)</span> is composed of the eigenvectors of <span class="math inline">\(A\)</span>. When <span class="math inline">\(A\)</span> is a <span class="math inline">\(2 \times 2\)</span> matrix, it is enough to find 2 linearly independent eigenvectors <span class="math inline">\(x\)</span> and <span class="math inline">\(y\)</span> for the matrix <span class="math display">\[X = \Biggl( \begin{array}{c|c} x_1 & y_1 \\ x_2 & y_2 \end{array} \Biggr)\]</span> to be invertible.</p>
<h2 id="eigenvectors">Eigenvectors</h2>
<p>The <strong>eigenvectors</strong> of a matrix <span class="math inline">\(A\)</span> are the <em>nonzero</em> vectors <span class="math inline">\(x\)</span> such that for an eigenvalue <span class="math inline">\(\lambda\)</span> of <span class="math inline">\(A\)</span>, <span class="math display">\[Ax = \lambda x.\]</span> If <span class="math inline">\(x\)</span> is an eigenvector, so is any <span class="math inline">\(\alpha x\)</span> for any scalar value <span class="math inline">\(\alpha\)</span>. If there are two linearly independent eigenvectors <span class="math inline">\(x\)</span> and <span class="math inline">\(y\)</span> associated to an eigenvalue, <span class="math inline">\(\alpha x + \beta y\)</span> is also an eigenvector. There is at least one eigenvector for each distinct eigenvalue, but there may be more than one when the eigenvalue is repeated.</p>
<p><strong>Example </strong> Distinct, real eigenvalues The matrix <span class="math display">\[A =\begin{pmatrix}
-1 & -2 \\
0 & 1
\end{pmatrix}\]</span></p>
<p>is upper-triangular; this is one of the eigenvalue-relealing shapes. The eigenvalues are <span class="math inline">\(-1\)</span> and <span class="math inline">\(1\)</span>. These are distinct eigenvalues, so each eigenvalue possesses a single eigenvector. The eigenvector <span class="math inline">\(x\)</span> associated to <span class="math inline">\(\lambda_1 = -1\)</span> is found by solving the eigensystem <span class="math display">\[Ax = (-1)x.\]</span></p>
<p>The unknown quantity <span class="math inline">\(x\)</span> appears on both sides of the equation. We can find a simpler form by noting that multiplying a vector by the identity matrix is neutral: <span class="math inline">\((-1)x = (-1) I x.\)</span> The eigenproblem becomes <span class="math display">\[\begin{aligned}
A x & = (-1) I x, \\
A x - (-1) I x = 0, \\
\bigl( A - (-1) I \bigr) x = 0,\end{aligned}\]</span></p>
<p>that is, the eigenvector is a nonzero solution of the linear system <span class="math inline">\(\bigl( A - \lambda I \bigr) x = 0\)</span>. In general, if a matrix <span class="math inline">\(B\)</span> is invertible, the only solution to <span class="math inline">\(Bx=0\)</span> is <span class="math inline">\(x = 0\)</span> (the vector of zeroes). But, by construction, <span class="math inline">\(A - \lambda I\)</span> cannot be invertible if <span class="math inline">\(\lambda\)</span> is an eigenvalue: its determinant is exactly the characteristic polynomial evaluated at one of its roots, so it is zero. This is why the eigensystem has nonzero solutions. Now, because <span class="math inline">\(A - \lambda I\)</span> is not invertible, this means that a least one of its rows is a linear combination of the others. For <span class="math inline">\(2 \times 2\)</span> matrices, this implies that the two rows are colinear, or redundant. For our example, the eigensystem reads <span class="math display">\[\begin{aligned}
\begin{pmatrix}
-1 - (-1) & -2 \\
0 & 1 - (-1)
\end{pmatrix}
\begin{pmatrix}
x_1 \\
x_2 \\
\end{pmatrix}
& =
\begin{pmatrix}
0 \\
0 \\
\end{pmatrix}, \\
\begin{pmatrix}
0 & -2 \\
0 & 2
\end{pmatrix}
\begin{pmatrix}
x_1 \\
x_2 \\
\end{pmatrix}
& =
\begin{pmatrix}
0 \\
0 \\
\end{pmatrix}.\end{aligned}\]</span></p>
<p>we immediately see that the two rows <span class="math inline">\((0,-2)\)</span> and <span class="math inline">\((0,2)\)</span> are colinear, with a factor <span class="math inline">\(-1\)</span>. This leads to an underdetermined system: <span class="math inline">\(0 x_1 + -2 x_2 = 0\)</span>. The solution is <span class="math inline">\(x_2 = 0\)</span> and we can take <span class="math inline">\(x_1\)</span> to be any value, save 0. We choose <span class="math inline">\(x = (1, 0)^t\)</span>.</p>
<p>For the eigenvalue <span class="math inline">\(\lambda_2 = +1\)</span>, the eigensystem reads: <span class="math display">\[\begin{aligned}
\begin{pmatrix}
-1 - (+1) & -2 \\
0 & 1 - (+1)
\end{pmatrix}
\begin{pmatrix}
y_1 \\
y_2 \\
\end{pmatrix}
& =
\begin{pmatrix}
0 \\
0 \\
\end{pmatrix}, \\
\begin{pmatrix}
-2 & -2 \\
0 & 0
\end{pmatrix}
\begin{pmatrix}
y_1 \\
y_2 \\
\end{pmatrix}
& =
\begin{pmatrix}
0 \\
0 \\
\end{pmatrix}.\end{aligned}\]</span></p>
<p>Again, the second row <span class="math inline">\((0,0)\)</span> can be neglected, and the solution is <span class="math inline">\(-2 y_1 + 2 y_2 = 0\)</span>, or <span class="math inline">\(y_1 = y_2\)</span>. It is customary to choose an eigenvector with norm 1. The <strong>norm</strong> of a complex-valued vector <span class="math inline">\(y = (y_1, y_2)^t\)</span> is <span class="math display">\[||y|| = \sqrt{y^*y} = \sqrt{\bar y_1 y_1 + \bar y_2 y_2} = \sqrt{|y_1|^2 + |y_2|^2}.\]</span></p>
<p>Here, the eigenvector is <span class="math inline">\(y = (y_1, y_1)^t\)</span>, so <span class="math inline">\(||y|| = \sqrt{|y_1|^2 + |y_1|^2} = \sqrt{2}\sqrt{|y_1|^2} = \sqrt{2}|y_1|.\)</span> Taking <span class="math inline">\(||y|| = 1\)</span> solves <span class="math inline">\(|y_1| = 1/\sqrt{2}.\)</span> This means that we could take a negative, or a complex value for <span class="math inline">\(y_1\)</span>, as long as the <span class="math inline">\(|y_1| = 1/\sqrt{2}.\)</span> Going for simplicity, we take <span class="math inline">\(y_1 = 1/\sqrt{2}\)</span>.</p>
<p><strong>Example </strong> Complex eigenvalues</p>
<p>The matrix <span class="math display">\[A =\begin{pmatrix}
0 & -1 \\
1 & 0
\end{pmatrix}\]</span> is <em>not</em> diagonal, so we have to compute the eigenvalues by hand. The trace of <span class="math inline">\(A\)</span> is zero, the determinant is <span class="math inline">\(0 - (1)(-1) = 1\)</span>, and the discriminant is <span class="math inline">\(-4\)</span>. A negative discriminant implies complex eigenvalues, <span class="math display">\[\lambda_{1,2} = \frac 12 \bigl( 0 \pm \sqrt{-4} \bigr) = \pm i.\]</span> For the eigenvalue <span class="math inline">\(\lambda_1 = +i\)</span>, the eigensystem reads: <span class="math display">\[\begin{aligned}
\begin{pmatrix}
- (+i) & -1 \\
1 & - (+i)
\end{pmatrix}
\begin{pmatrix}
x_1 \\
x_2 \\
\end{pmatrix}
& =
\begin{pmatrix}
0 \\
0 \\
\end{pmatrix}, \\
\begin{pmatrix}
-i & -1 \\
1 & -i
\end{pmatrix}
\begin{pmatrix}
x_1 \\
x_2 \\
\end{pmatrix}
& =
\begin{pmatrix}
0 \\
0 \\
\end{pmatrix}.\end{aligned}\]</span></p>
<p>The two rows <span class="math inline">\((-i,1)\)</span> and <span class="math inline">\((1,-i)\)</span> should be colinear, but this is not obvious with the complex coefficients. Multiplying the first row by <span class="math inline">\(i\)</span> gives <span class="math inline">\(i(-i, -1) = (- i^2, - i) = (-(-1), -i) = (1, -i)\)</span>, the second row, ok. Having confirmed that the system is indeed underdetermined, we can week a solution to <span class="math inline">\(-i x_1 - x_2 = 0\)</span>. Solving for <span class="math inline">\(x_2 = -i x_1\)</span>, we obtain the eigenvector <span class="math inline">\(x = (x_1, -i x_2)^t\)</span>. Normalization of <span class="math inline">\(x\)</span> imposes <span class="math display">\[||x|| = \sqrt{|x_1|^2 + |-ix_1|^2} = \sqrt{|x_1|^2 + |x_1|^2} = \sqrt{2}|x_1| = 1.\]</span> As in the previous example, we can choose <span class="math inline">\(x_1 = 1/\sqrt{2}.\)</span></p>
<p>The second eigenvectors, associated <span class="math inline">\(\lambda_2 = -i\)</span>, solves the eigensystem <span class="math display">\[\begin{aligned}
\begin{pmatrix}
- (-i) & -1 \\
1 & - (-i)
\end{pmatrix}