In this paper, we mainly focus on the integration of smart card based access to semantic web enabled multi-agent systems. Besides classical benefits such as smart card based authentication and authorization, integration of such a feature will make it possible for semantic web agents to take the personal knowledge stored as instances of a specific personal ontology in the smart cards into account and behave in a way that is more responsible to the individual requirements of the users. To integrate smart card based access to a semantic web agent, we need an agent plan specifically defined for that purpose. This plan will be responsible for both communicating with the smart card reader module and for semantically manipulating the personal knowledge that is transferred from the card. In the paper, we give the implementation level details for this plan. Another important aspect of the paper is that various alternatives for storing ontological knowledge on smart cards have been discussed based on some experimental results.