A Deep Inverse-Mapping Model for a Flapping Robotic Wing

We thanks the Reviewer. Comment: It seems the main contribution of this paper is the introduction of the adaptive spectrum layer in seq2sep prediction missions. Empirical effects are provided in the results (although the improvement provided in Table 2 seems not very promising). The innovation is relatively weak. The theoretical principle and intuitive explanation behind the learning architecture should be further abstracted and discussed. Answer: The innovation of this work is not only the introduction of the adaptive spectrum later, but also in the formation and learning of a new problem – an inverse mapping problem – in the context of an experimental system with complex and nonlinear dynamics. The improvement upon state-of-the-art architectures, such as Transformer, are sizeable (cf. our response to the review above), and obtained with a simpler architecture with improved scaling and latency (cf. response above). Per the Reviewer’s suggestion, we provide an intuitive explanation for the learning architecture in Section 3: “The essence of inverse-mapping modeling in this system lies in learning the relationship between desired output aerodynamic forces and the input wing kinematics that generate them. Intuitively, the input to the system is a time-sequence of desired aerodynamic forces that the wing should generate. The output is the full wing kinematics that, when applied to the wing, would result in the desired forces. The system learns the inverse mapping in two parallel backbones: learning the time-dependent (Seq2Seq) and frequency dependent (ASL) relationships between the forces. As a result, both the timestamps and frequencies that are relevant for the mapping are learned”. Comment: The implemented experiments are relatively simple. The real applied scenarios are usually more complicated, such as external disturbance, unknown dynamic model, and actuator uncertainty. The generalization problem of the developed learning architecture is not considered. To improve contributions, it is highly recommended to further include the learned inverse model in the online control framework, not just offline demonstration on the dataset of a flapping wing. Moreover, a comparison with a traditional control method (e.g., MPC) is needed. Answer: Unfortunately, the experiments performed here are not simple at all. Especially, while controlling the motor may appear trivial, it is utterly not so in these frequencies and accuracy constraints. While we would have loved to implement our system in a full-scale flying device, this task is complicated enough to place it beyond the scope of this work. Here, we demonstrate a critical and new module of a control system for flapping-wing systems that solves a NEW problem posed as an inverse mapping problem. Dealing with the many challenges suggested above would be the focus of future work. Comment: Some presentation errors. Every abbreviation that appears first in an article should be given its full name, such as FFT. All symbols employed in Fig 3 should be illustrated. A: Corrected. with thanks. Q: The equation (2) is called the one-step-ahead prediction model. Does \tau refer to the one-step size? However, \tau in equation (1) is defined as a period. It is confused. A: In both eqs. 1 and 2, \tau is the window size, as defined in the sentence following eq. 1. Specifically, \tau is size of the look-back window, which is the the model input, as usually defined by conditional probabilities (eqs. 1,2). Q: ...In real application, how to plan the desired force and how to implement the obtained attitude? A special control framework can be provided to illustrate the implementation of the learned inverse model. It is wondered if the uncertainty that exists in the control loop would influence the learning performance. A: In a flying flapping-robot, the desired force would be determined by the flight controller. In the currently existing robots, this stage is performed implicitly, via simple and manually-tuned PID control loops, whose output is wing kinematics, without directly determining the force (cf. Introduction). This implicit and heuristic approach does not exploit the full performance envelope of these systems, which pales in comparison to the performance of real insects. Namely, as we learn from observing insect flight, flapping wing systems have much broader capabilities and we believe that the current robots’ maneuverability, agility and robustness could be significantly improved using our explicit inverse-mapping. Our system will be combined as a module in such a flight controller (cf. Conclusion Section). Providing “a special control framework” is currently beyond the scope of this work – it is very challenging and we haven’t yet developed a fully flying device to test it on. As explained in the paper, the concept of inverse mapping is not limited to flapping-wing drones and translated to other systems as well.

We thank the Reviewer for noting the novelty, uniqueness and significance of this work. Below we refer to each on the the Reviewer's comments and suggestions. Q: Figures. A: Figures were edited and are now more readable. Thank you. Comment: There are many comparisons between different baseline methods, which is good. However, it would be beneficial to have a statistical test to show that the improvement in performance between Seq2Seq+ASL and the other baseline methods is statistically significant. In Figure 5, there is not that much difference between the Seq2Seq and Transformer models. A statistical test such as Mann Whitney U-test to compare the differences between the MAE losses. I leave the choice of statistical test to the authors discretion. A: We thank the referee for this very insightful comment. To provide more statistical power, we used the Wilcoxon signed rank test. Unlike the Mann Whitney U-Test, this test does not assume that the samples are independently distributed, which is more suitable for our case, since predictions were made on the same dataset. According to the Wilcoxon test, on our dataset, our system outperformed the Transform with a p-value of 0.06. Still, we note that our system outperformed the Transformer in 2/3 of the cases in our dataset, which is an indication for its performance. On the open-source dataset, our system outperformed the Transformer in 54.2% of the cases. The Wicoxon p-value in this case was 0.58, consistent with the notion these two architectures are equivalent on this dataset. These results have been added to the main text. Q: Please would the authors perform the statistical test to compare the statistical differences between MAE of their method against the baselines. A: Done, as detailed above. Q: Since ASL is a novel contribution of the authors, I would emphasize this to a greater degree in the abstract and the introduction. Is there a reason for not doing this? A: Corrected, with thanks. Q: Please could you point me to where you state the size of the dataset and evaluation sets used to compare results against the baselines. A: The sizes of the datasets are given in Table 1. The division is given in Line 418 of the manuscript: “Both datasets were randomly divided such that 75% of the events were used as a training set, 10% reserved for validation, and 15% for testing.” Q: I have a further optional suggestion for authors. Can the authors comment on the frequency spectrum present in the data? A: A sample spectrum is shown in the SI. Most power is concentrated at <210Hz. Q: This question is more one of curiosity, but may be of interest to other readers: I have a question about the RFFT and the IRFFT. How do you implement this in practice? I had assumed that most “fast” implementations are non-differentiable and I am curious which implementation you used. A: There are built-in differentiable layers in PyTorch for FFT/IFFT and RFFT/IRFFT, which we used here. The specific implementation will be available on GitHub upon publication of this work.

We thank the Reviewer for acknowledging the innovative aspects, implications and contribution of this work. Q: paper mentions different measurement types between datasets without explaining impact on model or justification A: The main difference between these dataset is their different sensor configurations and markedly different flow regimes, which our model learned successfully (Sections 4.1 and 4.2). Q: What is the sim2real gap for the real wing-driven robot? How to narrow the sim2real gap to make the research more useful. A: The term “sim2real gap” might not best describe the current setup, as it lacks any simulation. Learning was performed directly from experimental data. Beyond that, we expect minor differences between the desktop-mounted system and its airborne version, stemming from second order aerodynamic effects. Q: Can you scale to multiple degrees of freedom? how to evaluate the scaling? A: Our model deals with all 3 DOFs of a rigid wing connected at a hinge, while we actuate the stroke angle. In fact, most robotic applications control only phi, and the elevation angle is constrained to be zero, and wing-pitch angle is determined passively. Q: Can you scale to different geometry and material? How to evaluate? A: Our method handled data from two very different experimental systems, with different kinematics, materials and fluids, demonstrating its ability to scale across different systems. Q: What are the flight conditions? A: Estimated future flight parameters: flapping frequency ~20Hz, max speed ~5m/s, max rotation rate of ~200deg/sec, endurance time 5-10 minutes. Mass 30-50gr. Q: Any analysis of frequency selection? Why 100Hz/210Hz? A: These frequencies were based on examining the frequency content of the raw data and, particularly, the noise. All relevant information was contained in the band <210Hz (x10 of typical flapping frequency). Q: ... synchronization for different sensors, delay, data alignment A: An explanation added to Appendix A.6 Q: ...Processing delays? A: Delay is ~milliseconds, acceptable for wingbeat period of tens of ms, since control commands are calculated once per cycle. Q: Any stability analysis or guarantees for the control system? A: We expect that this module will improve the control performance of existing FW-MAVs, which currently rely on manually-tuned control loops, typically of the PID type. Q: How does the system handle disturbances or noise? A: The system provides a nominal mapping between desired force/torque and the wing motion that generates them. On its own, the system is not designed to handle flight disturbances -- this is done by the flight controller. Rather, our system will provide to the controller an “interface” between desired forces/torques and actuated wing motion. Q: theoretical justification for the model architecture choices A: First, The relevant data are temporal sequences of forces and angles, which are most suitable for seq2seq architectures. Second, RNN-based models have linear forward-pass time complexity, unlike transformer-based architectures, which use self-attention mechanisms with quadratic time complexity. This consideration is crucial for low-resource deployed devices, which cannot run parallel multi-core computation. Third, adding representation in frequency space is instrumental to modeling the relation between force inputs and wing kinematics, as both of them are periodic with similar frequencies. Q: generalizing to different Reynolds A: Tested Re span 1,000-50,000: the relevant flow regime for medium to large insects and, most importantly, flapping-wing MAVs (Sec 4.1). Q: performance of some basic NN A: We examined two linear models: LINEAR and NLINEAR, which underperformed compared with our system (Table2/Fig.5). Q: ... One core aspect of the paper is modeling aerodynamics, and similar work exists in the UAV field, such as NeuralFly. Therefore, this core contribution or novelty needs to be better clarified by the authors. A: First, flapping wing aerodynamics is orders of magnitude more complex than quadcopters. The forward problem of flapping wing has been extensively studied, but NO previous paper studied the inverse mapping (from desired forces to kinematics). Yet, this is the relevant problem for practical flight control of FW-UAVs. Second, recent papers that solve various problems in controlling quadcopters using ML/RL, e.g. under structural damage and external perturbations. NeuraFly is a neat example, in which an adaptive system learns the residual forces required to mitigate windy conditions. Our system is fundamentally different because it calculated the inverse mapping - from forces to wing kinematics. This problem is almost nonexistent in quadcopters because each rotor has only one DOF - rotation speed - and mapping between speed to force is trivial. Flapping wings are different -their aerodynamics is much more complex and hence the mapping. This point is emphasized in the revised intro.

Comment Re. Figure 5 and Table 2. Answer: We appreciate the suggestion. When we tried to accommodate the ablation study data, there was not enough room in the main text, especially following the post-review corrections. The value of Figure 5 is in showing the broader picture of the models performance, and Table 2, in our opinion, is a convenient summary of performance, in the format typical to the field. Comment: The Seq-2-Seq ASL model does not outperform the transformer on the open source dataset. But does perform better on the authors dataset. An explanation for this would greatly help the contribution of this paper. Answer: on the openSrc dataset, performance of the two models is equivalent in the mean metric: 0.1130 for our model vs. 0.1123 for the Transformer model. In the median metric, our model outperformed the Transformer by 11%. In this particular setting, Fig. 5 shows that seq2seq+ASL has more outliers than the Transformer, which explains the difference between the mean and median metrics. Clarification has been added to the caption 5. On our dataset, the seq2seq+ASL outperformed the Transformer by 10.4% (mean) and 15% (median). Relatively, our system outperformed the Transformer in 2/3 of the test samples. Equality important to the performance per-se, we note that our model inference is significantly faster than the Transformer model (new figure in Appendix + comment below). Thus, even though the models are equivalent on one of the two datasets, our model inference time makes it practical for onboard computing, much more than the Transformer. A new paragraph has been added to the Results section. Comment Re. the abstract Answer: Corrected as suggested. Question: Around 1000 experiments were performed for the hyperparameter search of the Seq-2-Seq ASL model. Was the same search conducted for other models in Table 2 ? Answer: Yes. Details about this procedure have been added to Section A3 in the revised Appendix. Question: What is the effect of scaling this model on the MAE ? Considering the goal of this effort is to deploy on highly compute constrained platforms, it would be interesting to see if this model scales better than a transformer. Answer: We thank the Reviewer for this question. A new figure showing the scaling of both the validation-MAE and the inference time with the number of parameters is now included in the Appendix (Section A.4). These results explain the selected configurations of seq2seq+ASL (200k parameters) and the Transformer model (50k parameters) analyzed in this work. Interestingly, while seq2seq+ASL has x4 more parameters than the Transformer, seq2seq+ASL has x10 shorter inference time, which makes it more practical for operational use given the flapping frequencies of current FW-MAVs. Thank you.