The earth observation data has increased significantly, over the last decades; NASA has 18 Earth observation satellites on orbit carrying 80 sensors, as of April 2003. About 3 terabytes of data are collected daily, and transmitted to Earth receiving stations. The data exploitation and dissemination methods have not kept pace with the huge data acquisition rate. The products distributed by the agencies are often not in a readily usable form by, the non-science community, and need further processing at the user level. The lack of content and semantic based interactive information searching, and retrieval capabilities from the archives is another important issue to be addressed in this context. In this paper we propose a framework based on a concept-based model using domain-dependant ontologies where the basic concepts of the domain are identified first and generalized later depending upon the level of reasoning required for executing a particular query. We employ an unsupervised segmentation algorithm to extract homogeneous regions and calculate primitive descriptors for each region based on color, texture and shape. The primitive descriptors are described quantitatively by middle level object ontology. The learning phase is applied at this stage. It associates the middle level descriptors to the concepts in the higher-level ontology by means of a nonlinear Support Vector Machine (SVM) method. These associations are grouped into models specific to a semantic class and used for querying. Also interactive querying is provided by means of a region based relevance feedback method. A methodology, to execute complex queries by, the integration of an inference engine is discussed. We also intend to extend the system to carry out data exploratory, tasks in a peer-to-peer environment.