Whenever I see a NetLogo based paper, my suspicion is immediately raised. It's not that multiagent simulations are never good models of reality, it's that they are so hard to get right and so easy to shoot yourself in the foot with and not notice. Nonlinear dynamics are hard.
Usually you write a simple model, it does something pathological like diverging to one extreme, you maybe tune the parameters a bit until it does something that fits better with your expectations, then you publish. But the behavior you observed is still pathological, dependent on fine tuning of the model parameters, and has nothing to do with the relationship between the model and reality.
I especially panicked when I saw people on HN arguing we should drop all those SIR-based models for Covid-19 and use multiagent simulations instead.
Also, NetLogo obfuscates from you that usually what you're trying to model is a very simple one-dimensional equation that could be written in three lines of Python, and that you could see the problems with immediately if you'd look at the actual equation.
>But the behavior you observed is still pathological, dependent on fine tuning of the model parameters, and has nothing to do with the relationship between the model and reality.
Basically, a mirror of modern DNN approaches: data driven correlations and hyperparemter tuning. Main difference are that with MAS, humans are usually tuning (this is changing though) vs DNN approaches, the tuning is algorithmically driven.
The one benefit I would claim is that with these MAS, you at least know which base assumptions you're really tweaking to converge on states of interest. The question is (as always) if you've accurately: reduced the problem and represented principle interactions in the model. Given the complexity of real world systems MAS try to model, I think this is optimistically incredibly challenging and more realistically, nearly impossible to achieve currently. I think it's still a useful modeling approach that should be explored though I feel it's still in it's infancy compared to numeric, analytic, statistical, etc. models. I think if its taken the surge of computing infrastructure to drive the current successes we're seeing in DNN, it may take an equally larger surge to see improvements in MAS.
Not at all similar in my opinion. Hyperparameters don't usually directly affect the model the DNN implements, they mostly affect the speed of convergence.
Stuff like sensitivity analysis is both easy to do and part of standard operating procedures when working with DNNS, but not when modeling with NetLogo.
Any modeling approach can be implemented/used in an incorrect way. The so-called analytical approaches also often make very specific assumptions on functional forms to be able to obtain analytical solutions. By performing extensive sensitivity analysis one can reduce the drawbacks you mention. Of course, this needs to be done, and you are probably right that many papers don't do this. But that is hardly a problem with the modeling technique, which can be quite effective if used appropriately.
I was exposed to this as a teenager some 20 years ago (when it was called StarLogo) and it was one of the main reasons why I got interested in collective behavior and emergence.
I've also been lucky enough to meet some of the people who worked on this during my career...
The power of community multi-agent simulations lie in participatory environments. Every geospatially dispersed team shares their collected data and eventually this is how the world gets solved.
I wish I had more time to participate! I noticed an interesting Human-in-the-Loop Learning (HILL) challenge running BattleSnake with AWS SageMaker. It's a well-known, time-tested result. Agents + Humans consistently outperform either humans or agents alone ;)
Battlesnake Challenge: A Multi-agent Reinforcement Learning Playground with Human-in-the-loop
The Santa Fe Institute's Complexity Explorer offers a course called Introduction to Agent-Based Modeling that uses NetLogo. Here's a link to an archive of the most recent course.
I used NetLogo at one point in my education. It was a lot of fun. Maybe too much fun. So easy to get sucked in to tweaking a parameter here and there and watching the same initial conditions evolve in different directions. Simulation researchers must have to be constantly on guard against having too much fun in their jobs.
Usually you write a simple model, it does something pathological like diverging to one extreme, you maybe tune the parameters a bit until it does something that fits better with your expectations, then you publish. But the behavior you observed is still pathological, dependent on fine tuning of the model parameters, and has nothing to do with the relationship between the model and reality.
I especially panicked when I saw people on HN arguing we should drop all those SIR-based models for Covid-19 and use multiagent simulations instead.
Also, NetLogo obfuscates from you that usually what you're trying to model is a very simple one-dimensional equation that could be written in three lines of Python, and that you could see the problems with immediately if you'd look at the actual equation.
This is a good example: https://arxiv.org/pdf/1802.07068.pdf
My own review is in Hebrew, but it seems like this is a good one: http://joshuaballoch.github.io/luck-in-life-still-misunderst...