High Dimensional Data, MSRI, and San Francisco in 2018; Reflections

This article is originally published at https://ntguardian.wordpress.com

Last fall my adviser alerted me to the MSRI workshop on high-dimensional data and suggested I may be interested. I applied and was accepted to participate. Thus, from July 9th to July 20th I stayed in San Francisco (for the first time in my life), living in the dorms of UC Berkeley and attending the workshop. I got to experience San Francisco’s legendary weather (escaping Salt Lake City’s triple-digit heat) while learning mathematics. I enjoyed the experience and wanted to share it.

The Mathematical Sciences Research Institute (MSRI) is a non-profit organization for promoting and facilitating mathematical research. MSRI functions independently of UC Berkeley, even being located just above and off the campus. Many students attending the summer schools ride a bus from the University campus to MSRI along a windy wooded road until reaching the top of the hill. Thus, MSRI overlooks the San Francisco bay and provides an excellent view at tea time.

MSRI enjoys a lovely locale, nestled in woodlands. While one could take a bus to and from the campus, many enjoy walking up and down the mountain trails connecting MSRI with the rest of Berkeley. (I was not one of these people, sadly.) One day we got to appreciate just how arboreal MSRI’s location is when a roving pack of turkeys attacked the building one morning.

The facility houses a kitchen, dining areas, an area for tea and mingling, and of course an auditorium and library. Naturally, workshop lecturers delivered their talks in the auditorium, and I spent considerable time in the library browsing the collection. (More on that later.) All told, it was a wonderful place to visit.

The workshop effectively consisted of two weeks that covered different topics only loosely connected to one another. The first week was about compressive sensing, while the second week covered machine learning topics and programming.

Jeff Blanchard effectively lead the first week’s topic. In short, we learned about solving the equation , where , , , and is much smaller than . We also require that is sparse; that is, most of the entries of are zero, except for a few. The setup is illustrated below (with this image from Wikipedia).

We saw a number of theorems about the problem and when it can be solved, in addition to algorithms for solving the problem and proofs they worked. Prof. Blanchard presented his proofs as digestible and pleasant exercise. I enjoyed the exercises and was willing to do them even outside of the exercise sessions, back in the dorms. (This unfortunately was not always the case.) In fact, just seeing how Prof. Blanchard took a proof and converted it into exercises enlightened me. In the past I have wondered how I can retain more from readings, such as books or papers, where no exercises are offered. Prof. Blanchard’s exercises suggested how I could generate my own exercises from a reading.¹

However, I don’t know when I, personally will ever use the material I studied. Heck, I barely understood what compressive sensing is for! We saw pictures where compressed sensing techniques were able to reconstruct a picture after 90% of the original picture’s information was deleted (set to zero). I asked Prof. Blanchard to explain the pictures to me, and it seems that they were constructed using methods that resembled what we had seen but were much more involved. I suppose this is fine, but I wanted an example application that I could implement using the techniques I had learned. I wish I was provided a relatively simple “real world” problem, a data set, where I could have tried out the compressive sensing techniques. I like the mathematics and I want to see it and study it, but in applied mathematics I appreciate seeing how the methods are used on “real” data (even if that “real” data is fictitious).

(The above image is from here.)

Even then, I don’t expect to ever work on compressive sensing problems. That’s not MSRI’s fault; I’m a single minded person who cares little for the world of mathematics outside of probability/stochastics, statistics (especially econometrics), economics/finance, and statistical modeling. Having said that I don’t feel like I wasted my time.

Prof. Blanchard left Friday, July 13th, and after a weekend off we returned the next week to study machine learning. I (and many of the other participants I spoke with) were less impressed with the second week. Monday felt like it was filled with data science talks we were tired of hearing, where problems were presented or methods shown with fancy pictures and impressive results and no discussion of details. Sometimes this is fine, but the crowd listening to the talks consisted of aspiring mathematicians who came to a workshop to dive into the theory of new ideas and grapple with the mathematics that allow these constructs to function. One high-level talk would have been fine if it were followed by two talks diving deeper into that subject. Then on Tuesday we had crash courses in Python for scientific computing and data analysis. People listening fell into one of two groups: either the listener already knew about Python, NumPy, SciPy and Matplotlib and learned almost nothing (I fell into this group; after all, I have authored four video courses on using Python for data analysis, though I was shaky on how to use Matplotlib) or didn’t know anything about any of these things and learned nothing because the talks were too fast. On the third day, Dr. Blake Hunter from Microsoft gave talks that did go deeper into the problems that SVMs, logistic regression, topic modeling, etc. solve, and I enjoyed his talks. For instance, I learned why the eigenvalues give information about clusters in spectral clustering; I had not seen that theory before (or at least did not understand it the last time I saw it). I feel like if Dr. Hunter’s talks were extended out to most of the week (like with Prof. Blanchard’s talks on compressive sensing), he could have gone into more details (he frequently said he wanted to skip over details and technicalities when asked about them) and I would have felt like I got more out of that week. But the day ended in an exercise session where Dr. Hunter told us to find a data set and do something with it. I downloaded a Kaggle data set that turned out to be really big, too big to be done in exercise sessions, and when I got back to the dorms I was not willing to reconsider the problem. I would have appreciated more direction. Thursday was talks that either seemed useless or were so mired in unfamiliar notation I understood nothing (but I was impressed, whatever it was she was talking about). Friday consisted of participants giving two-minute presentations on their research (I bombed mine, underestimating how quickly two minutes goes) and a panel discussion. The latter was enlightening; it convinced me that I need to attend more conferences.

I think the workshop should have been more focused. It would have been best if it were either about compressed sensing or machine learning, rather than half of one and half of the other. If that were not possible, the machine learning week should have been much more focused. Perhaps the second week was doomed because it was the second week and people’s motivation was starting to wane; however, I think the problems were deeper than that. We would have appreciated in depth discussions all throughout the week building ideas and exploring their theory (again, this was a mathematics workshop; we craved theory) rather than a smattering of loosely related data science talks. Of course, a workshop like this cannot be expected to be polished; cutting edge ideas are not polished. If the talks were more concentrated, though, they may have had a stronger effect.

All told, I think the workshop would have been better if a single idea were explained well enough that a participant could come away literate enough to start reading, understanding, and maybe even writing research papers on the topic.

A number of students from universities all over the nation and orgins from all over the world attended, including former Baltimore Ravens guard and center John Urschel. You can watch a video about him below.

I should have done more networking; there were many intelligent people at the workshop and I could have perhaps made friends. Instead, I may have been the most antisocial person there (or at least that’s how I felt).

It didn’t help that the Thursday prior to the workshop I came down with gastroenteritis. I rode the plane with no issues but the next day (the first day of the workshop) I was not feeling well and was in no mood to interact with people. Tuesday was the last day of recovery but I was so nervous (and embarrassed) about my condition that I didn’t want to interact with anyone any more than I needed. That likely sealed my isolation, in addition to having some social anxiety. I’m perfectly willing to give a presentation in front of a crowd with no worries, but one-on-one contact with people that isn’t in a professional function always stirs butterflies in me (yes, I’m painfully single too; I can’t talk to new women if my life depended on it).

I think I need a long time to get comfortable with people, and I did later become more willing to interact with others. I particularly enjoyed going to a nearby bar, the Tap Haus, having drinks and playing games with the other attendees. Perhaps if the workshop went on longer I would have acclimated even better, but I doubt I’ll see any of the other attendees ever again.

I enjoyed the library at MSRI more than I should. It’s not a particularly large library, but it was a cozy library devoted to mathematics filled with interesting books. I looked forward to leaving the lecture and going to the library, finding a book, and enjoying the quiet, pleasant atmosphere. (This is probably another reason I was antisocial; I wanted to be with the books.)

We were required to attend a library orientation, and in the orientation I learned about the library’s facilities and also about MathSciNet, a tool I had never heard of and likely will utilize a lot in the future. The books in the library are listed alphabetically by author. Admittedly this is not a great way to index the library’s contents; organization by subject seems superior. Surprisingly, though, I enjoyed the fact that books were indexed by name. The result was my seeing titles that I never would have seen since I would never go to that section of a library willingly. Furthermore, since I couldn’t look at books by subject, I looked for books by authors I knew.

In the spring semester of this year, I had a student ask me for suggestions for a statistical history topic. The student was taking Prof. Andrejs Treibergs History of Mathematics class (Prof. Treibergs was also my instructor for my first statistics classes, MATH 3070 and MATH 3080) and needed to write a paper. I initially suggested the student investigate William Gosset and his work at Guinness (yes, the brewer) that lead to the creation of the -test, but this topic was already taken by another student. At a loss, I suggested that the student read David Salsburg’s The Lady Tasting Tea, a book about the history of statistics. I recommmended the book without having read it myself, and after the student borrowed a copy from the University library I bought a copy for myself. The book was a pleasure to read, and the idea of a “history of mathematical ideas” class like Prof. Treibergs’ class has swam around in my mind in recent months (I even bought a copy of the class’s textbook for myself while I was in San Francisco, from the legendary Moe’s Books).

So when I found The History of Statistics in the 17th and 18th Centuries, I was well primed. The book consists of lectures written by Prof. Karl Pearson, considered by many to be the father of mathematical statistics. The book was published posthumously, edited by Karl Pearson’s son, Egon Pearson, the “Pearson” from the Neyman-Pearson lemma. “Neyman” is Prof. Jerzy Neyman, professor at UC Berkeley and the man largely responsible for the confidence interval and hypothesis testing framework now taught in introductory (frequentist) statistics courses. I mention this because many of the books in the library, including The History of Statistics in the 17th and 18th Centures, appear to be from his personal collection.

(Ronald Fisher’s book, Statistical Methods for Research Workers; Fisher is another important figure in statistics history and unrelenting critic of Jerzy Neyman; notice that this book, an early classic in statistics, appears to have been owned by Jerzy Neyman.)

I read this book for hours almost every day at MSRI; when lectures were over, I would go to the library, take a seat on the couch, and read. I did not really know there was a history of statistics prior to the 19th century, yet Karl Pearson delightfully describes the ideas, the methods, the people, and the historical context as a good historian should.

The book is long; I only managed to get through the third chapter, after Karl Pearson describes the life and work of Edmond Halley and Caspar Neumann. Nevertheless it’s a delight to read.

Consider, for example, Pearson’s reprinting of an account of how Dr. William Petty, contemporary of John Graunt (perhaps the first demographer, authoring the first life table, and whom Prof. Pearson identifies as the father of statistics) purportedly revived a woman from death.

Anne Greene ….. was, at a Sessions held in Oxford, arraigned, condemned, and on Saturday the 14th of December last, brought forth to the place of the Execution [for adultery]; where, after signing of a Psalme, and something said in justification of her self, as to the fact for which she was to suffer, and touching the lewdnesse of the Family wherein she lately lived, she was turn’d off the ladder, hanging by the neck for the space of almost halfe an houre, some of her friends in the mean time thumping her on the breast, others hanging with all their weight upon her leggs; sometimes lifting her up, and then pulling her downe againe with a suddaine jerke, thereby the sooner to dispatch her out of her paine: insomuch that the Under Sherriffe, fearing lest thereby they should breake the rope, forbad them to doe so any longer. At length when everyone thought she was dead, the body being taken downe, and put into a coffin, was carried thence into a prive house, where some Physicians had appointed to make a Dissection. The coffin being opened, she was observed to breathe, and in breathing (the passage of her throat ebing streightened) obscurely to ruttle: which being perceived by a lusty fellow that stood by, he (thinking to doe an act of charity in ridding her of the small reliques of a painfull life) stamped severall times on her breast and stomack with all the force he could. Immediately after, there came in Dr Petty of Brasen-nose-Colledge, our Anatomy Professor, and Mr Thomas Willis of Christ-Churh, and whose comming, which was about 9 o’clock in the morining, she yet persisted to ruttle as before, laying all the while streched out in the coffin in a cold room and season of the yeare. They perceiving some life in her, as well for the humanity as their Profession sake, fell presently to act in order to her recovery. First, having caused her to be held up in the coffin, they wrenched open her teeth, first were fast set, and poured into her mouth some hot and cordiall spirits; whereupon she ruttled more than before, and seemed obscurely to cough: then they opened her hands (her fingers also being stifly bent) and ordered some to rub and chafe the extreme parts of her body …..

Whilst the Physicians were thus busie in recovering her to life, the Under-Sheriffe was solliciting the Governor and the rest of the Justices of the Peace for the obtaining her Reprieve, that in case she should for that present be recovered fully to life, shee might not be had backe again to Execution. Whereupon those worthy Gentlemen, considering what had happened, weighing all circumstances, they readily apprehended the hand of God in her preservation, and being willing rather to co-operate with divine providence in saving her, than to overstrain justice by condemning her to double shame and sufferings, they were pleased to grant her a Reprieve.

Thus, within the space of a month, was she wholly recovered: and in the same room where her body was to have been dissected for the satisfaction of a few, she became a great wonder, being revived to the satisfaction of multitudes that flocked thither daily to see her.

I don’t know about you, but for a book on the history of statistics, I’d call that a wild story.

I enjoyed the book so much that I ordered a personal copy, and plan to finish the book. If a student comes to me looking for a mathematical history topic, I can now offer my own ideas for papers.

I may have rambled through this post, wandering from topic to topic, but I can accept that. I wanted to share my thoughts and ideas.

My trip to MSRI got me more interested in my work as a Ph.D. student. When the academic year begins again in late August, I hope to start attacking topics and publishing papers, getting a research program going. MSRI helped give me that taste of the academy that I’ve been missing.

Here’s hoping I can visit UC Berkeley and MSRI again someday.

I have created a video course published by Packt Publishing entitled Training Your Systems with Python Statistical Modeling, the third volume in a four-volume set of video courses entitled, Taming Data with Python; Excelling as a Data Analyst. This course discusses how to use Python for machine learning. The course covers classical statistical methods, supervised learning including classification and regression, clustering, dimensionality reduction, and more! The course is peppered with examples demonstrating the techniques and software on real-world data and visuals to explain the concepts presented. Viewers get a hands-on experience using Python for machine learning. If you are starting out using Python for data analysis or know someone who is, please consider buying my course or at least spreading the word about it. You can buy the course directly or purchase a subscription to Mapt and watch it there.

If you like my blog and would like to support it, spread the word (if not get a copy yourself)! Also, stay tuned for future courses I publish with Packt at the Video Courses section of my site.

Thanks for visiting r-craft.org
This article is originally published at https://ntguardian.wordpress.com
Please visit source website for post related comments.