With the advances in financial technology (FinTech) in recent years, the finance industry has attempted to enhance the efficiency of their services through technology. The financial service most commonly desired by people is a robo-advisor, which is a virtual expert that advises people on how to make good decisions related to financial market trading. Some mathematical models may be difficult to apply to prediction problems involving multiple cross correlation. To overcome this issue, the rules of and implicit correlation between the collected data must be determined through algorithms to forecast the future market situation. By combining financial information from the financial market, we apply an information fusion approach to collect data. Also, a relational model can be developed using a deep neural network, which can be represented using a graph structure, to determine the real market situation. However, the structural information of the graph may be lost if a traditional deep learning module is used. Therefore, in this paper, we present a visual-question-answering-like deep learning fusion model based on the graph structure and attention mechanism. This model was used to study the interaction between the time-series data of various financial variables. The relationships among financial commodities were formulated, and the graph representation was learned through graph neural networks. Moreover, the importance of each commodity was determined through the attention mechanism. The proposed method improved the accuracy of trend and volatility prediction up to 6% on S&P500 and developed a pairs trading application.