This paper introduces a novel graph signal processing framework for building\ngraph-based models from classes of filtered signals. In our framework,\ngraph-based modeling is formulated as a graph system identification problem,\nwhere the goal is to learn a weighted graph (a graph Laplacian matrix) and a\ngraph-based filter (a function of graph Laplacian matrices). In order to solve\nthe proposed problem, an algorithm is developed to jointly identify a graph and\na graph-based filter (GBF) from multiple signal/data observations. Our\nalgorithm is valid under the assumption that GBFs are one-to-one functions. The\nproposed approach can be applied to learn diffusion (heat) kernels, which are\npopular in various fields for modeling diffusion processes. In addition, for\nspecific choices of graph-based filters, the proposed problem reduces to a\ngraph Laplacian estimation problem. Our experimental results demonstrate that\nthe proposed algorithm outperforms the current state-of-the-art methods. We\nalso implement our framework on a real climate dataset for modeling of\ntemperature signals.\n