I want to get deeper into the Machine Learning(theory and application in Finance). I want to ask how relevant are complex analysis and functional analysis as a basis for Machine Learning? Do I need to learn these subjects or should I concentrate myself on other topic (if so, on which?)
Solved – Complex analysis, Functional analysis for deeper understanding Machine Learning
machine learningpac-learning
Related Solutions
For (1), as ebony1 suggests, there are several incremental or on-line SVM algorithms you could try, the only thing I would mention is that the hyper-parameters (regularisation and kernel parameters) may also need tuning as you go along as well, and there are fewer algorithmic tricks to help with that. The regularisation parameter will almost certainly benefit from tuning, because as the amount of training data increases, the less regularisation that is normally required.
For (2) you could try fitting a one-class SVM to the training data, which would at least tell you if the data were consistent with the classes you do know about, and then classify as "unknown" if the output of the one-class SVM was sufficiently low. IIRC libsvm has an implementation of one-class SVM.
For (3) if you just use 1-v-1 class component SVMs, then you just need to make three new SVMs, one for each of the unknown-v-known class combinations, and there is no need to retrain the others.
HTH
Yes, I think tag prediction is an interesting one and one for which you have a good shot at "success".
Below are some thoughts intended to potentially aid in brainstorming and further exploration of this topic. I think there are many potentially interesting directions that such a project could take. I would guess that a serious attempt at just one or two of the below would make for a more than adequate project and you're likely to come up with more interesting questions than those I've posed.
I'm going to take a very wide view as to what is considered machine learning. Undoubtedly some of my suggestions would be better classified as exploratory data analysis and more traditional statistical analysis. But, perhaps, it will help in some small way as you formulate your own interesting questions. You'll note, I try to address questions that I think would be interesting in terms of enhancing the functionality of the site. Of course, there are many other interesting questions as well that may not be that related to site friendliness.
- Basic descriptive analysis of user behavior: I'm guessing there is a very clear cyclic weekly pattern to user participation on this site. When does the site get the most traffic? What does the graph of user participation on the site look like, say, stratified by hour over the week? You'd want to adjust for potential changes in overall popularity of the site over time. This leads to the question, how has the site's popularity changed since inception? How does the participation of a "typical" user vary with time since joining? I'm guessing it ramps up pretty quickly at the start, then plateaus, and probably heads south after a few weeks or so of joining.
- Optimal submission of questions and answers: Getting insight on the first question seems to naturally lead to some more interesting (in an ML sense) questions. Say I have a question I need an answer to. If I want to maximize my probability of getting a response, when should I submit it? If I am responding to a question and I want to maximize my vote count, when should I submit my answer? Maybe the answers to these two are very different. How does this vary by the topic of the question (say, e.g., as defined by the associated tags)?
- Biclustering of users and topics: Which users are most alike in terms of their interests, again, perhaps as measured by tags? What topics are most similar according to which users participate? Can you come up with a nice visualization of these relationships? Offshoots of this would be to try to predict which user(s) is most likely to submit an answer to a particular question. (Imagine providing such technology to SE so that users could be notified of potentially interesting questions, not simply based on tags.)
- Clustering of answerers by behavior: It seems that there are a few different basic behavioral patterns regarding how answerers use this site. Can you come up with features and a clustering algorithm to cluster answerers according to their behavior. Are the clusters interpretable?
- Suggesting new tags: Can you come up with suggestions for new tags based on inferring topics from the questions and answers currently in the database. For example, I believe the tag [mixture-model] was recently added because someone noticed we were getting a bunch of related questions. But, it seems an information-retrieval approach should be able to extract such topics directly and potentially suggest them to moderators.
- Semisupervised learning of geographic locations: (This one may be a bit touchy from a privacy perspective.) Some users list where they are located. Others do not. Using usage patterns and potentially vocabulary, etc, can you put a geographic confidence region on the location of each user? Intuitively, it would seem that this would be (much) more accurate in terms of longitude than latitude.
- Automated flagging of possible duplicates and highly related questions: The site already has a similar sort of feature with the Related bar in the right margin. Finding nearly exact duplicates and suggesting them could be useful to the moderators. Doing this across sites in the SE community would seem to be new.
- Churn prediction and user retention: Using features from each user's history, can you predict the next time you expect to see them? Can you predict the probability they will return to the site conditional on how long they've been absent and features of their past behavior? This could be used, e.g., to try to notice when users are at risk of "churn" and engage them (say, via email) in an effort to retain them. A typical approach would shoot out an email after some fixed period of inactivity. But, each user is very different and there is lots of information about lots of users, so a more tailored approach could be developed.
Related Question
- Solved – Building background for machine learning for CS student
- Solved – Machine learning applications in number theory
- Machine Learning – How to Work with Weighted and Complex Survey Data
- Solved – Textbooks about reproducing kernel Hilbert space approach to machine learning
- Solved – Two worlds collide: Using ML for complex survey data
- Solved – clear intersection of chaos theory and machine learning
Best Answer
I would say that the most important pre-requisites to Machine Learning are Linear Algebra, Optimization (both numerical and theoretical) and Probabilities.
If you read at the details of the implementations of common machine learning algorithms (I have in mind the LASSO, Elastic Net, SVMs) the equations heavily relies on various identities (dual form of an optimization problem, various formulae stemming from linear algebra) and the implementation requires you to be familiar with techniques such as gradient descent.
Probabilities are a must have both in the PAC Learning Framework and every time you study tests.
Then, only then, functional analysis can come in handy. Especially when you are studying kernels (and use representation theorems).
Regarding complex analysis, I am not aware of major use of important theorems stemming from this field in machine learning (someone correct me if I am wrong).