Conversational AI combines natural language processing (NLP) with machine and deep learning models so that people can interact in human-like manner with the digital devices. The quality of social interactions can be additionally improved by utilizing the physical presence of the robot and prompting context derived from the robot's hardware. Therefore, we propose a modular software architecture to integrate Conversational AI into Socially Assistive Robots (SARs). It follows a flow-based approach with shared repositories and direct or message-based input/output channels. We conducted two experiments to test the architecture's modularity and adaptability. The first experiment focused on the performance of different NLP cloud services and their associated modules, while the second experiment tested the integration of the Conversational AI in two different SARs - NAO and Pepper. Our experimental results demonstrate that the architecture is general enough to be applied for various SARs and different NLP use cases.